Application of Deep Learning Algorithm for Web Shell Detection in Web Application Security System

Abstract

A web shell is a script executed on a web server, often used by hackers to gain control over an infected server. Detecting web shells is challenging due to their complex behavior patterns. This research focuses on using a deep learning approach to detect web shells on the ISB Atma Luhur web server, aiming to develop a model capable of precise detection. By training the model with labeled PHP files, malicious web shells are distinguished from benign files. The study is crucial for enhancing the server's security, preventing hacker attacks, and safeguarding sensitive data. Through preprocessing techniques such as opcode extraction and feature selection, useful pattern recognition for web shell detection is achieved. Training deep learning models like CNN and RNN with LSTM on processed data leads to accuracy evaluation using classification metrics. The CNN model demonstrates superior performance in detection, emphasizing the effectiveness of deep learning for web shell detection. The research contributes to enhancing security in web-based applications, protecting against cyber threats like web shells.