Explainable AI Framework for Proactive Cybersecurity Defense

Mobile reading ranks No.4 on the permeate proportion of the mobile Internet applications and is increasing gradually. Understanding user behaviors of mobile reading is important for service provider. However, few work has been done on reading analysis. In this paper, we provide an approach to solve this problem based on cloud computing platform including HDFS and Apache Pig. Our study is carried out based on the data collected from the ISP network which covered an entire city in southern China. We presented the feature of traffic and behavior in three aspects. First, we compared the service condition of several popular websites in mobile reading field and found the basic law that traffic distribution fits. Second, we recognize user sessions via time threshold method and found some interesting phenomenon about session duration. Finally, we break user actions down into discrete categories and summarized the characteristics of each reading application. We compared user acceptance of different applications from three perspectives, which are user loyalty, user behavior and cross-user distribution respectively.

In this paper, we extract the characteristics of WeChat traffic and propose approaches to identify WeChat traffic in cellular data network. WeChat communication mechanisms are discussed. The traffic and usage pattern of Video Call service provided by WeChat are studied from massive traffic data using Spark, differently from previous methods. We analyze the features of WeChat Video Call service, a Voice over Internet Protocol (VoIP) application in three aspects, which are (i) daily/weekly usage pattern, (ii) traffic/usage distribution, (iii) conversation time distribution. Our analysis has two important features. Firstly, the massive mobile subscriber data we used in our experiments was collected from a commercial Internet Service Provider (ISP) covering an entire province in Northern China ensuring that the results reflect the real characteristics of service in question in cellular network. Secondly, we investigate that the WeChat Video Call usage times fit with the power law distribution. Our results are important for cellular network operators and service providers to realize WeChat traffic identification methods and user behavior of Video Call, which imply information for optimization of their services.

By research of the current network traffic idenfication methods and typical network user behavior analysis methods，a online network user behavior analysis model has been designed and implemented. In order to achieve internal network user behavior real-time monitoring and online analysis purposes.

With the explosion of knowledge and the high‐speed dissemination of information, people’s desire for knowledge and information is getting stronger and stronger. At the same time, the updating of knowledge and information is going on at an unprecedented speed. The traditional teaching mode is affected by time and space. Its limitations have become more and more prominent; the traditional classroom teaching has been unable to meet the existing teaching needs. At present, there are many methods for the analysis of network user behavior, such as statistical methods, association analysis methods, and clustering algorithms. Among them, clustering algorithms are more widely used in network user behavior analysis, which is closely related to the unsupervised and high efficiency of clustering algorithms. This paper combines the advantages of clustering algorithm in network user behavior analysis and, on the basis of the existing clustering algorithm research, proposes an improved algorithm for the analysis of online intelligent teaching art resources, so as to obtain the law of online behavior of student users in campus network. Provide some help for students’ Internet management and network optimization. Finally, summarize and put forward the concept of intelligent teaching and design and implement an online intelligent teaching art resource platform based on cluster analysis algorithm. Studies have shown that the average number of transactions processed by the platform per second is 65.21, which can well simulate real information query use cases. The transaction processing time of the platform will eventually stabilize between 30 s and meet the performance requirements.

Recently, aligning users among different social networks has received significant attention. However, most of the existing studies do not consider users' behavior information during the aligning procedure and thus still suffer from poor learning performance. In fact, we observe that social network alignment and user behavior analysis can benefit from each other. Motivated by such an observation, we propose to jointly study the social network alignment and user behavior analysis problem in this paper. We design a novel framework named BANANA-RGB. In this framework, to capture users' multi-scale behavior information in each social network, we train a variant of the hierarchical periodic memory network with personalized memorization. To leverage behavior analysis for social network alignment, we design a tensor fusion network-based alignment component to improve the performance. To further leverage social network alignment for behavior analysis, we design a gating-based cross-network behavior fusion component to integrate users' behavior information in different social networks based on the alignment result. We iteratively train the above two components to make the two tasks benefit from each other. Extensive experiments on real-world datasets demonstrate that our proposed approach outperforms the state-of-the-art methods.

On monitoring and predicting mobile network traffic abnormality

Machine learning (ML) is being used to improve intrusion detection mechanisms and identification in cyber security. Network data volume scaling (with the help of Machine learning) — Automated analysis and pattern recognition for large amounts of network-data, thereby detection of anomalies / potentially malicious activities that escape current rule-based techniques. By training ML models on historical data, these models can learn benign network behavior as well the anomalies in them that may result from malicious activities. The purpose of this essay/report is to begin taking a look under the hood at how ML can be used for security threat detection and analysis in-networking on an ongoing basis. This paper covers the automation of ML algorithms to enhance network security. Given the current state of electronic threats and their evolution, traditional security methods are typically insufficient. ML can analyze large volumes of data, learn patterns from it and this makes it suitable to complement network defense mechanisms. It then discusses different ML applications in network cybersecurity: intrusion detection, anomaly detection, spam and malware analysis; which the paper characterizes. It analyzes the potential, benefits and constrains of major ML methods in network security like supervised learning; unsupervised learning and reinforcement-learning. Finally, this paper represents recent progress in the use and impact of ML techniques along with case studies.The paper discusses the existing difficulties in the field such as the necessity for datasets and the vulnerability of machine learning models to adversarial attacks.T he paper also highlights avenues for exploration by focusing on developing scalable security solutions based on machine learning that are resilient and flexible.The goal of this examination is to offer both researchers and industry professionals valuable perspectives into the opportunities and obstacles linked to utilizing machine learning, in the domain of network security. ML methods have potential to improve network security by addressing the challenges posed by the increasing cyber threats that traditional security measures struggle to combat effectively over time. One key strength of ML lies in its capacity to analyze datasets and identify intricate patterns efficiently. The research paper delves into applications of ML in enhancing network security. The list covers security tools like Intrusion Detection Systems (IDS) examining malware and phishing attempts as well as anomalies in network activity and user behavior analysis (UEBA). The study explores both supervised and unsupervised learning methods. How they are used for quick threat detection and response in real time scenarios.You will find case studies and recent developments that showcase the implementation and effectiveness of these strategies.In addition the article delves into the obstacles linked to using machine learning techniques in network security including the necessity, for datasets, The paper's goal is to give an enlightening summary to scholars and practitioners about how machine learning can be applied to network security in order to provide solutions that are robust, adaptive, and scalable. To this end, it touches on several relevant aspects. One is the threat posed by adversarial attacks on the sorts of models that are likely to be used in this context. Another is the imperative, deriving from both adversarial threat and model drift, that models needed in this context be available in a form usable for continuous update. Keywords: Network Security, machine learning (ML), Intrusion Detection Systems (IDS), entity behavior analytics (UEBA).

Botnet detection via mining of traffic flow characteristics

For many decades, telephone network traffic has been characterized by Poisson call arrivals and exponentially distributed call durations (with a mean of about 3 minutes). In contrast, recent work in network traffic data analysis and modeling has demonstrated that data network traffic is best characterized in terms of burstiness and connection times that range over a wide range of time scales (from milliseconds to minutes and hours); in addition, data network traffic has been observed to undergo constant and often radical changes within short periods in time, due to a constantly changing user population, the emergence of so-called “killer applications” (e.g., the Web and other multimedia services in today’s Internet), new networking technologies, etc. Historically, the areas of network traffic data analysis and modeling have suffered from a severe “drought” of data. However, more recently, this drought has been replaced by a “flood” of traffic measurements from today’s high-speed communication networks that keeps increasing in volume and speed. As a result of these changes, network research (in particular, traffic analysis and traffic modeling) has started to adopt concepts that have a long tradition in the physical sciences but have been all but ig-nored in the social sciences and in the mainstream statistics literature. On January 10, 1997, Walter Willinger from AT&T Labs—Research illustrated some of these concepts and showed how they apply to modern network traffic analysis and modeling. He pointed out their implications on traffic engineering and performance analysis of current and future high-speed networks. Finally he outlined new areas of research in the mathematical and statistical sciences that result from these changes and are of practical importance for network research.

It is great significance for network administrators, researchers, service providers and users to accurately identify network bandwidth occupied by what application and who. It is the foundation for network planning, troubleshooting, intrusion detection, traffic monitoring, accounting management, and user behavior analysis. Traffic analysis based on full data has high requirements of storage space and CPU processing power. Traffic analysis based on NetFlow requires network equipment to support NetFlow. The availability of former and the accuracy of the latter are challenged when data traffic up to 1Gbp or higher.In this paper, we consider user preferences, server address database, application layer protocol features and NetFlow technology when designing the application layer protocol identification algorithms, which can improve the availability and accuracy of identification, experimental results it works well.

Botnet detection based on traffic behavior analysis and flow intervals

In a high-speed network, traffic monitoring modules should be compact in size to fit into a fast but small memory (e.g., SRAM). We propose two compact algorithms for network traffic monitoring and analysis, for the purposes of per-flow traffic measurement and long-duration flow detection. The proposed schemes are based on the data structure of a virtual vector that was recently invented, but limited to the purpose of estimating spread value. We found that the virtual vector can be applied to a range of different problems in the area of network traffic monitoring and analysis. In this article, we propose a counting virtual vector that counts the number of packets for per-flow traffic measurement. For long-duration flow detection, we observe that the attackers can easily evade the previous work and propose a new detection scheme to catch even evasive flows. Through experiments on real Internet traffic traces, we show that the proposed schemes outperform previous work or make up for its weaknesses.

Effective network traffic identification has important significance for network monitoring and management, network planning and user behavior analysis. In order to select and extract the most effective attribute as well as explore the inherent correlation between the attributes of network traffic. We proposed a new network traffic identification method based on deep factorization machine (DeepFM) which can classify and do correlation analysis simultaneously. Specifically, we first embed the feature vector into a joint space using a low-rank matrix, then followed by a factorization machine (FM) which handle the low-order feature crosses and a neural network which can learn the high- order feature crosses, finally the low-order feature crosses and high-order feature crosses are fused and give the classified result. We validate our method on Moore dataset which is widely used in network traffic research. Our results demonstrate that DeepFM model not only have a strong ability of network traffic identification but also can reveal some inherent correlation between the attributes.

Recently, aligning users among different social networks has received significant attention. However, most of the existing studies do not consider users’ behavior information during the aligning procedure and thus still suffer from the poor learning performance. In fact, we observe that social network alignment and behavior analysis can benefit from each other. Motivated by such an observation, we propose to jointly study the social network alignment problem and user behavior analysis problem. We design a novel end-to-end framework named BANANA. In this framework, to leverage behavior analysis for social network alignment at the distribution level, we design an earth mover’s distance based alignment model to fuse users’ behavior information for more comprehensive user representations. To further leverage social network alignment for behavior analysis, in turn, we design a temporal graph neural network model to fuse behavior information in different social networks based on the alignment result. Two models above can work together in an end-to-end manner. Through extensive experiments on real-world datasets, we demonstrate that our proposed approach outperforms the state-of-the-art methods in the social network alignment task and the user behavior analysis task, respectively.

PurposeWith the rapid development of digital humanities, some digital humanities platforms have been successfully developed to support digital humanities research for humanists. However, most of them have still not provided a friendly digital reading environment and practicable social network analysis tool to support humanists on interpreting texts and exploring characters’ social network relationships. Moreover, the advancement of digitization technologies for the retrieval and use of Chinese ancient books is arising an unprecedented challenge and opportunity. For these reasons, this paper aims to present a Chinese ancient books digital humanities research platform (CABDHRP) to support historical China studies. In addition to providing digital archives, digital reading, basic search and advanced search functions for Chinese ancient books, this platform still provides two novel functions that can more effectively support digital humanities research, including an automatic text annotation system (ATAS) for interpreting texts and a character social network relationship map tool (CSNRMT) for exploring characters’ social network relationships.Design/methodology/approachThis study adopted DSpace, an open-source institutional repository system, to serve as a digital archives system for archiving scanned images, metadata, and full texts to develop the CABDHRP for supporting digital humanities (DH) research. Moreover, the ATAS developed in the CABDHRP used the Node.js framework to implement the system’s front- and back-end services, as well as application programming interfaces (APIs) provided by different databases, such as China Biographical Database (CBDB) and TGAZ, used to retrieve the useful linked data (LD) sources for interpreting ancient texts. Also, Neo4j which is an open-source graph database management system was used to implement the CSNRMT of the CABDHRP. Finally, JavaScript and jQuery were applied to develop a monitoring program embedded in the CABDHRP to record the use processes from humanists based on xAPI (experience API). To understand the research participants’ perception when interpreting the historical texts and characters’ social network relationships with the support of ATAS and CSNRMT, semi-structured interviews with 21 research participants were conducted.FindingsAn ATAS embedded in the reading interface of CABDHRP can collect resources from different databases through LD for automatically annotating ancient texts to support digital humanities research. It allows the humanists to refer to resources from diverse databases when interpreting ancient texts, as well as provides a friendly text annotation reader for humanists to interpret ancient text through reading. Additionally, the CSNRMT provided by the CABDHRP can semi-automatically identify characters’ names based on Chinese word segmentation technology and humanists’ support to confirm and analyze characters’ social network relationships from Chinese ancient books based on visualizing characters’ social networks as a knowledge graph. The CABDHRP not only can stimulate humanists to explore new viewpoints in a humanistic research, but also can promote the public to emerge the learning interest and awareness of Chinese ancient books.Originality/valueThis study proposed a novel CABDHRP that provides the advanced features, including the automatic word segmentation of Chinese text, automatic Chinese text annotation, semi-automatic character social network analysis and user behavior analysis, that are different from other existed digital humanities platforms. Currently, there is no this kind of digital humanities platform developed for humanists to support digital humanities research.