Trigger-based malware samples exhibit untrustworthy aims provided that certain environmental conditions are satisfied. We have categorized trigger-based malware behaviours into two classes, i.e., invasive and evasive. Invasive behaviour is concerned with malicious activities, while malware uses evasive behaviours for self-defence. In this article, we propose a greedy incremental approach for detecting invasive trigger-based malicious behaviour. The method proceeds by identifying and supplying resources required by malign samples. Trigger-based behaviours vary depending on environmental conditions. Such behaviours can be modelled as directed graphs, where each node represents a state, and edges denote invocation of a specific event. We define each state in the graph as atomic behaviour, represented using API calls, having legitimate functionality. This atomic behaviour transforms into harmful action if augmented with additional APIs. A well-known challenge with behavioural analysis is the proactive identification of malicious activities before devices are compromised. Timely prediction of malicious behaviours without damaging the systems can be fulfilled by defining them as atomic actions. To reckon a sub-behaviour as atomic, we propose a novel algorithm. Besides, a new likelihood statistical significance test, Htest, is suggested to extract frequent subgraphs of graphs, representing malware family signature. The main message underlying Htest is that the lesser the number of subgraphs exhibiting benign behaviour, the higher the chances of the sub-graph being chosen as malware family signature. Experiments conducted on 2320 real malware samples demonstrate a 42% reduction in the number of signatures than the state-of-the-art methods, namely CDG and CMQDG. Also, 3% and 1.3% improvement in path coverage compared to the state-of-the-art methods, namely Pytrigger and GoldenEye.
Read full abstract