Large Network Data Research Articles

Perbandingan Support Vector Machine, Random Forest Classifier, dan K-Nearest Neighbour dalam Pendeteksian Anomali pada Jaringan DDos

A Distributed Denial of Service (DDoS) attack poses a serious threat to network security and can disrupt online services by overwhelming the target server with excessive traffic. Effective detection of DDoS attacks requires a system capable of identifying anomalies in network traffic. In this context, Machine Learning (ML) offers an effective approach for classification and anomaly detection. However, different ML algorithms have varying strengths and weaknesses when processing large and complex network data. Therefore, this study aims to evaluate the performance of three ML algorithms: Support Vector Machine (SVM), Random Forest Classifier (RFC), and K-Nearest Neighbors (KNN) in detecting DDoS anomalies. The dataset used consists of 225,745 data points with 85 attributes that describe various characteristics of network traffic, such as destination port, flow duration, packet count, and packet size. This dataset is classified into two classes, BENIGN and DDoS, representing normal traffic and DDoS attacks, respectively. Evaluation is performed using several performance metrics, including accuracy, precision, recall, MCC (Matthews Correlation Coefficient), F-Measure, ROC Area, PRC Area, True Positive Rate (TPR), and False Positive Rate (FPR). The results show that the Random Forest Classifier (RFC) delivers the best performance with an accuracy of 99.99%, precision of 99.98%, recall of 100%, and a very low FPR of 0.02%. This is followed by the Support Vector Machine (SVM) with an accuracy of 99.91%, and the K-Nearest Neighbor (KNN) with an accuracy of 99.98%. All three algorithms demonstrate strong performance in detecting DDoS anomalies, with RFC slightly outperforming others in terms of consistency and higher classification capability. The findings of this study provide valuable insights for selecting the best algorithm to detect DDoS attacks in networks.

Understanding Co-Offending Patterns and Criminal Mentorship Using Police Registry Data

ABSTRACT The current study examines registration data from the Dutch police pertaining to 2,757 individuals and 10 different crime categories, at the offense, individual, and network levels of analysis. Moreover, a novel estimate is introduced which identifies individuals’ tendency to operate as mentors or mentees in criminal networks. We find co-offending to be conditional on the type of crime and to typically involve groups of two to three offenders. Moreover, we find that men and those aged 18–25 are most likely to co-offend and that co-offenders typically do not show a preference for particular co-offenders. Individuals with central positions in the co-offending network were found to be older and to offend more frequently with a higher average of co-offenders per offense, than those less central. Lastly, we find the novel estimate to be a promising tool for identifying mentor and mentee roles in large quantitative network data. The results of the current study emphasize the importance of studying co-offending on different analytical levels, and to differentiate co-offending by different crime types, network positions, and individual roles, to better inform theory and policy.

Explainable Spark-based PSO Clustering for Intrusion Detection

Given the exponential growth of available data in large networks, the existence of rapid, transparent and explainable intrusion detection systems has become of high necessity to effectively discover attacks in such huge networks. To deal with this challenge, we propose a novel explainable intrusion detection system based on Spark, Particle Swarm Optimization (PSO) clustering and eXplainable Artificial Intelligence (XAI) techniques. Spark is used as a parallel processing model for the effective processing of large-scale data, PSO is integrated for improving the quality of the intrusion detection system by avoiding sensitive initialization and premature convergence of the clustering algorithm and finally, XAI techniques are used to enhance interpretability and explainability of intrusion recommendations by providing both micro and macro explanations of detected intrusions. Experiments are conducted on several large collections of real datasets to show the effectiveness of the proposed intrusion detection system in terms of explainability, scalability and accuracy. The proposed system has shown high transparency in assisting security experts and decision-makers to understand and interpret attack behavior.

SCALABILITY ISSUES IN DISTRIBUTED DATA PROCESSING FOR FOGGY IT INFRASTRUCTURES

This paper addresses scalability issues in distributed data processing for fog IT infrastructures. The analysis identified large data volumes, network congestion, device heterogeneity, and network instability as key issues. Possible next steps are considered, such as implementing effective strategies and using the latest technologies to address these issues. Proposals are made to improve fog IT infrastructures through the introduction of innovative solutions and research in this area. The abstract emphasizes the need for an integrated approach and cooperation between scientists and practitioners to overcome scalability issues in fog IT infrastructures. In addition, attention is drawn to the importance of data security in the context of fog IT infrastructures and the need to develop appropriate measures to ensure it.

OHDSI Standardized Vocabularies-a large-scale centralized reference ontology for international data harmonization.

The Observational Health Data Sciences and Informatics (OHDSI) is the largest distributed data network in the world encompassing more than 331 data sources with 2.1 billion patient records across 34 countries. It enables large-scale observational research through standardizing the data into a common data model (CDM) (Observational Medical Outcomes Partnership [OMOP] CDM) and requires a comprehensive, efficient, and reliable ontology system to support data harmonization. We created the OHDSI Standardized Vocabularies-a common reference ontology mandatory to all data sites in the network. It comprises imported and de novo-generated ontologies containing concepts and relationships between them, and the praxis of converting the source data to the OMOP CDM based on these. It enables harmonization through assigned domains according to clinical categories, comprehensive coverage of entities within each domain, support for commonly used international coding schemes, and standardization of semantically equivalent concepts. The OHDSI Standardized Vocabularies comprise over 10 million concepts from 136 vocabularies. They are used by hundreds of groups and several large data networks. More than 8600 users have performed 50 000 downloads of the system. This open-source resource has proven to address an impediment of large-scale observational research-the dependence on the context of source data representation. With that, it has enabled efficient phenotyping, covariate construction, patient-level prediction, population-level estimation, and standard reporting. OHDSI has made available a comprehensive, open vocabulary system that is unmatched in its ability to support global observational research. We encourage researchers to exploit it and contribute their use cases to this dynamic resource.

Enhancing intrusion detection performance using explainable ensemble deep learning.

Given the exponential growth of available data in large networks, the need for an accurate and explainable intrusion detection system has become of high necessity to effectively discover attacks in such networks. To deal with this challenge, we propose a two-phase Explainable Ensemble deep learning-based method (EED) for intrusion detection. In the first phase, a new ensemble intrusion detection model using three one-dimensional long short-term memory networks (LSTM) is designed for an accurate attack identification. The outputs of three classifiers are aggregated using a meta-learner algorithm resulting in refined and improved results. In the second phase, interpretability and explainability of EED outputs are enhanced by leveraging the capabilities of SHape Additive exPplanations (SHAP). Factors contributing to the identification and classification of attacks are highlighted which allows security experts to understand and interpret the attack behavior and then implement effective response strategies to improve the network security. Experiments conducted on real datasets have shown the effectiveness of EED compared to conventional intrusion detection methods in terms of both accuracy and explainability. The EED method exhibits high accuracy in accurately identifying and classifying attacks while providing transparency and interpretability.

Community detection based on community perspective and graph convolutional network

Community detection is an essential topic in network analysis, which aims to divide a network into multiple subgraphs to mine potential information. However, most existing approaches tend to separate representation learning from clustering and fail to detect overlapping communities. Furthermore, these methods do not take a community perspective and cannot effectively capture information at the community level. In this paper, we propose a new community detection method based on the community perspective and graph convolution network (CPGC) to address these limitations in attributed networks without prior label information. First, through the Bernoulli-Poisson model, CPGC combines representation learning and clustering and can be used for overlapping communities. Second, we modify the classical graph convolution to enhance the discriminability of node representations, making them more suitable for increasingly large network data. Finally, we propose a novel community perspective similarity and introduce cross-community modularity to leverage community-level information. These improvements enable CPGC to be community-oriented and explore potential community structures more accurately. Experiments on various real-world networks show that CPGC can achieve state-of-the-art results in nonoverlapping or overlapping communities.

Blockchain-based privacy and security model for transactional data in large private networks

Cyberphysical systems connect physical devices and large private network environments in modern communication systems. A fundamental worry in the establishment of large private networks is mitigating the danger of transactional data privacy breaches caused by adversaries using a variety of exploitation techniques. This study presents a privacy-preserving architecture for ensuring the privacy and security of transaction data in large private networks. The proposed model employs digital certificates, RSA-based public key infrastructure, and the blockchain to address user transactional data privacy concerns. The model also guarantees that data in transit remains secure and unaltered and that its provenance remains authentic and secure during node-to-node interactions within a large private network. The proposed model has increased the encryption speed by about 17 times, while the decryption process is expedited by 4 times. Therefore, the average overall acceleration obtained was 16.5. Both the findings of the security analysis and the performance analysis demonstrate that the proposed model can safeguard transactional data during communications on large private networks more effectively and securely than the existing solutions.

Decision Key-Value Feature Construction for Multihoming Big Data Network

The random forest algorithm under the MapReduce framework has too many redundant and irrelevant features, low training feature information, and low parallelization efficiency when dealing with multihoming big data network problems, so parallelism is based on information theory, and norms is proposed for random forest algorithm (PRFITN). In this paper, the technique used first builds a hybrid dimensional reduction approach (DRIGFN) focused on information gain and the Frobenius norm, successfully reducing the number of redundant and irrelevant features; then, an information theory feature is offered. This results in the dimensionality-reduced dataset. Finally, a technique is suggested in the Reduce stage. The features are grouped in the FGSIT strategy, and the stratified sampling approach is employed to assure the information quantity of the training features in the building of the decision tree in the random forest. When datasets are provided as key/value pairs, it is common to want to aggregate statistics across all objects with the same key. To acquire global classification results and achieve a rapid and equal distribution of key-value pairs, a key-value pair redistribution method (RSKP) is used, which improves the cluster’s parallel efficiency. The approach provides a superior classification impact in multihoming large data networks, particularly for datasets with numerous characteristics, according to the experimental findings. We can utilize feature selection and feature extraction together. In addition to minimizing overfitting and redundancy, lowering dimensionality contributes to improved human interpretation and cheaper computing costs through model simplicity.

GPU-based, interactive exploration of large spatiotemporal climate networks.

This paper introduces the Graphics Processing Unit (GPU)-based tool Geo-Temporal eXplorer (GTX), integrating a set of highly interactive techniques for visual analytics of large geo-referenced complex networks from the climate research domain. The visual exploration of these networks faces a multitude of challenges related to the geo-reference and the size of these networks with up to several million edges and the manifold types of such networks. In this paper, solutions for the interactive visual analysis for several distinct types of large complex networks will be discussed, in particular, time-dependent, multi-scale, and multi-layered ensemble networks. Custom-tailored for climate researchers, the GTX tool supports heterogeneous tasks based on interactive, GPU-based solutions for on-the-fly large network data processing, analysis, and visualization. These solutions are illustrated for two use cases: multi-scale climatic process and climate infection risk networks. This tool helps one to reduce the complexity of the highly interrelated climate information and unveils hidden and temporal links in the climate system, not available using standard and linear tools (such as empirical orthogonal function analysis).

Classification of Large Social Twitter Network Data Using R

The development of social networks has altered computer science research. Now, a vast amount of data is available via Twitter, Facebook, emails, and IoT. (Internet of Things). So, storing and analyzing these data has become very difficult for academics. Conventional frameworks have been ineffective in processing massive amounts of data. R is an open-source programming language designed for large-scale data analysis with higher accuracy.
 Additionally, it offers the chance to implement the R programming language. This essay examines the application of R to classify sizable social network data. The Naive Bayes method is used to categorize massive amounts of Twitter data. The experiment has demonstrated that a sizable portion of data may be adequately classified with positive outcomes utilizing the R framework.

Real-Time Intrusion Detection and Prevention System for 5G and beyond Software-Defined Networks

The philosophy of the IoT world is becoming important for a projected, always-connected world. The 5G networks will significantly improve the value of 4G networks in the day-to-day world, making them fundamental to the next-generation IoT device networks. This article presents the current advances in the improvement of the standards, which simulate 5G networks. This article evaluates the experience that the authors gained when implementing Vodafone Romania 5G network services, illustrates the experience gained in context by analyzing relevant peer-to-peer work and used technologies, and outlines the relevant research areas and challenges that are likely to affect the design and implementation of large 5G data networks. This paper presents a machine learning-based real-time intrusion detection system with the corresponding intrusion prevention system. The convolutional neural network (CNN) is used to train the model. The system was evaluated in the context of the 5G data network. The smart intrusion detection system (IDS) takes the creation of software-defined networks into account. It uses models based on artificial intelligence. The system is capable to reveal not previously detected intrusions using software components based on machine learning, using the convolutional neural network. The intrusion prevention system (IPS) blocks the malicious traffic. This system was evaluated, and the results confirmed that it provides higher efficiencies compared to less overhead-like approaches, allowing for real-time deployment in 5G networks. The offered system can be used for symmetric and asymmetric communication scenarios.

A distributed message passing algorithm for the capacitated directed Chinese postman problem

Message passing is a class of extremely powerful distributed iterative algorithms based on probabilistic graphic model, in which little computation performed per iteration on minimal data structure . As a prototypical message-passing algorithm, belief propagation (BP) algorithm has wide applications in various fields of coding theory, machine learning and combinatorial optimization . Due to its distributed and iterative nature, BP algorithm can run effectively and fast on large data networks. In this paper, we study the behavior of Min-Sum BP algorithm for the capacitated directed Chinese postman problem ( CDCP ). We derive the iterative process of message passing for solving CDCP . As the main result, for any weighted digraph G of size n , if the weight on each edge is nonnegative integral, then our algorithm converges to the optimal solution of CDCP after O ( w ∗ n 2 ) iterations, provided that CDCP has a unique optimal solution, where w ∗ = max { w e : e ∈ E ( G ) } . • Message passing is a class of extremely powerful distributed iterative algorithms based on probabilistic graphic model. • Due to its distributed and iterative nature, belief propagation (BP) algorithm can run effectively and fast on large data networks. • BP-based algorithms can also run fast on a large data network in synchronous circumstances. • Min-Sum BP algorithm could converge to the optimal solution of the capacitated directed Chinese postman problem after O ( w m a x n 2 ) iterations.

Stingray Venom Proteins: Mechanisms of Action Revealed Using a Novel Network Pharmacology Approach.

Animal venoms offer a valuable source of potent new drug leads, but their mechanisms of action are largely unknown. We therefore developed a novel network pharmacology approach based on multi-omics functional data integration to predict how stingray venom disrupts the physiological systems of target animals. We integrated 10 million transcripts from five stingray venom transcriptomes and 848,640 records from three high-content venom bioactivity datasets into a large functional data network. The network featured 216 signaling pathways, 29 of which were shared and targeted by 70 transcripts and 70 bioactivity hits. The network revealed clusters for single envenomation outcomes, such as pain, cardiotoxicity and hemorrhage. We carried out a detailed analysis of the pain cluster representing a primary envenomation symptom, revealing bibrotoxin and cholecystotoxin-like transcripts encoding pain-inducing candidate proteins in stingray venom. The cluster also suggested that such pain-inducing toxins primarily activate the inositol-3-phosphate receptor cascade, inducing intracellular calcium release. We also found strong evidence for synergistic activity among these candidates, with nerve growth factors cooperating with the most abundant translationally-controlled tumor proteins to activate pain signaling pathways. Our network pharmacology approach, here applied to stingray venom, can be used as a template for drug discovery in neglected venomous species.

Large Geographical Area Aerial Surveillance Systems Data Network Infrastructure Managed by Artificial Intelligence and Certified over Blockchain: A Review

This paper proposes an aerial data network infrastructure for Large Geographical Area Surveillance Systems. The work presents a review of previous works from the authors, existing technologies in the market, and other scientific work, with the goal of creating a data network supported by Autonomous Tethered Aerostat Airships used for sensor fixing, a drones deployment base, and meshed data network nodes installation. The proposed approach for data network infrastructure supports several independent and heterogeneous services from independent, private, and public companies. The presented solution employs Edge Artificial Intelligence (AI) systems for autonomous infrastructure management. The Edge AI used in the presented solution enables the AI management solution to work without the need for a permanent connection to cloud services and is constantly fed by the locally generated sensor data. These systems interact with other network AI services to accomplish coordinated tasks. Blockchain technology services are deployed to ensure secure and auditable decisions and operations, which are validated by the different involved ledgers.

A Survey on Data-driven Network Intrusion Detection

Data-driven network intrusion detection (NID) has a tendency towards minority attack classes compared to normal traffic. Many datasets are collected in simulated environments rather than real-world networks. These challenges undermine the performance of intrusion detection machine learning models by fitting machine learning models to unrepresentative “sandbox” datasets. This survey presents a taxonomy with eight main challenges and explores common datasets from 1999 to 2020. Trends are analyzed on the challenges in the past decade and future directions are proposed on expanding NID into cloud-based environments, devising scalable models for large network data, and creating labeled datasets collected in real-world networks.

Automated Detection of Instability-Inducing Channel Geometry Transitions in Saint-Venant Simulation of Large-Scale River Networks

A new sweep-search algorithm (SSA) is developed and tested to identify the channel geometry transitions responsible for numerical convergence failure in a Saint-Venant equation (SVE) simulation of a large-scale open-channel network. Numerical instabilities are known to occur at “sharp” transitions in discrete geometry, but the identification of problem locations has been a matter of modeler’s art and a roadblock to implementing large-scale SVE simulations. The new method implements techniques from graph theory applied to a steady-state 1D shallow-water equation solver to recursively examine the numerical stability of each flowpath through the channel network. The SSA is validated with a short river reach and tested by the simulation of ten complete river systems of the Texas–Gulf Coast region by using the extreme hydrological conditions recorded during hurricane Harvey. The SSA successfully identified the problematic channel sections in all tested river systems. Subsequent modification of the problem sections allowed stable solution by an unsteady SVE numerical solver. The new SSA approach permits automated and consistent identification of problem channel geometry in large open-channel network data sets, which is necessary to effectively apply the fully dynamic Saint-Venant equations to large-scale river networks or for city-wide stormwater networks.

21 Navigating Controversial Therapies for Stevens-Johnson Syndrome and Toxic Epidermal Necrolysis Syndrome Using Large Database Analysis

Abstract Introduction Stevens-Johnson Syndrome (SJS) and Toxic Epidermal Necrolysis Syndrome (TENS) are part of a spectrum of autoimmune conditions, which cause epidermal detachment and keratinocyte necrosis. Due to the rare incidence, a dramatic heterogeneity in treatment algorithms and prolonged discussion of controversial therapies has ensued. We queried a large national data network to better understand how these therapies are actually implemented and the relative impact on survival. Methods The TriNetX Global Health Research Network (41 healthcare organizations) was queried using ICD-10 codes (L51.1 - 51.3) to identify patients diagnosed with SJS or TENS from 2000 to 2020. A treatment frequency map was constructed to determine the most common treatment groupings, and cohorts were indexed based on the most frequent isolated and combined therapy algorithms: systemic steroids (SS), diphenhydramine (DH), cyclosporine (CS), intravenous immunoglobulin (IVIG) and tumor necrosis factor alpha inhibitors (TNFαi). Cohorts were case matched for demographics against a restricted control group (RC; patients who did not receive any of the above-mentioned therapies) and an unrestricted control group (UC, all patients diagnosed with SJS or TENS) to evaluate mortality risk, survival probability, and to uncover Type II error. Results Cohorts were UC (6,196), RC (2,248), SS+DH (3,459), SS (1,269), SS+CS (1,554), DH (479), CS (52), IVIG (10) and TNFi (10). The treatment map showed 36.3% of patients did not receive any of the listed therapies. Of those that did 48.2% initially got SS, 24.3% got DH, 15.4% got SS+DH and 3% got SS+CS. Patients who received SS had a 8.51% mortality risk and 2.84% risk reduction compared to UC (p = 0.017). However, the Hazard Ratio (HR) was 0.80 (95% CI: 0.57, 1.23) showing no survival advantage. Compared to RC risk reduction decreased to 0.47% (p = 0.667). SS+DH showed a risk reduction of 1.13% compared to UC (p = 0.113; HR 0.89, 95% CI: 0.69, 1.16), but this advantage resolved when compared to RC. Similarly, SS+CS had a risk reduction of 2.12% compared to UC (p = 0.039; HR 0.80, 95% CI: 0.58, 1.09), which decreased to 0.07% (p = 0.947) when compared to RC. DH and CS had no significant risk reduction (p = 0.25 - 1.00) or survival advantage. IVIG and TNFαi were underpowered for analysis. Conclusions The most common treatments for patients diagnosed with SJS or TENS are systemic steroids, diphenhydramine, or a combination of the two. Unfortunately, none of the above therapies confer a significant survival advantage. Furthermore, some therapies raised concern for Type II errors when the control group is not adjusted for alternative therapies.

Design and Implementation of an Efficient Communication System for Collecting Sensor Data in Large Scale Sensors Networks

Design and Implementation of an Efficient Communication System for Collecting Sensor Data in Large Scale Sensors Networks

Big data network security index correlation measure based on the fusion of modified two order cone programming model

The existing technology can not meet the needs of large data network security index measurement in the era of big data. New theories and methods need to be explored to support the application of large data. In this paper, an improved second-order cone programming model (MTOCPM) is used as the data expression vector. The power spectral density feature of large data is extracted from a large number of noisy and fuzzy data. The second-order cone programming model of large data information flow is constructed by rough concept lattice method. Combining with the finite convergence of the second-order cone programming model algorithm, the reliability of each clustering sample is measured, and the accurate search of data clustering centre is realised. Experiments show that the method based on MTOCPM fusion can not only greatly reduce the communication cost between network nodes, but also improve the global measurement accuracy by about 10%.