User Behavior Audit System of Real-time Web Log by Spark

Abstract

With the rapid development of information technology, single sign-on authentication systems have been adopted by organizations to protect their information management systems and other service platforms. With the single sign-on method, users just log in the authentication systems once to access all the information in systems, which are more convenience, but accompanied by user permissions vulnerabilities and information security risks. Users’ browsing behaviors reflect the users’ usage habits, purposes, and work contents. Risks of information leakage can be reduced by auditing the users’ browsing behaviors. After analyzing the web logs of several service systems, we designed a user behavior audit system based on the Spark computing engine, which takes the real-time web log file as the data set. A possible preprocessing method for real-time web log data stream is proposed to extract users’ web session data. we used the Spark computing engine and the corresponding analysis to mine users’ behavior patterns. The online real-time detection and offline detection of user behavior are realized, and users are divided into three levels: normal, abnormal, and risky. The system administrators or auditors can view the calculation results of each stage online, manage the parameter configuration of the algorithm online, and adjust the user’s access right in real-time.