Large-scale Computer Networks Research Articles

A High-Performance Infrastructure for Remote Sensing Data Applications Using HPC Paradigms

Individuals and businesses are currently involved in the administration of remote sensing data that was previously handled only by government agencies. There is a lot more information in remote sensing data than go through the eye, and retrieving it is time-consuming and computationally expensive. Clusters, distributed networks, and specialized hardware devices are essential to speeding up remote sensing data extraction calculations. HPC advances in remote sensing applications are examined in this research. High-performance computing (HPC) concepts for instance FPGAs and GPUs as well as large-scale and heterogeneous computer networks are examined (GPUs). Using HPC paradigms, remote sensing applications are examined in these sections.

Parameter Estimation of Binned Hawkes Processes

A key difficulty that arises from real event data is imprecision in the recording of event time-stamps. In many cases, retaining event times with a high precision is expensive due to the sheer volume of activity. Combined with practical limits on the accuracy of measurements, binned data is common. In order to use point processes to model such event data, tools for handling parameter estimation are essential. Here we consider parameter estimation of the Hawkes process, a type of self-exciting point process that has found application in the modeling of financial stock markets, earthquakes and social media cascades. We develop a novel optimization approach to parameter estimation of binned Hawkes processes using a modified Expectation-Maximization algorithm, referred to as Binned Hawkes Expectation Maximization (BH-EM). Through a detailed simulation study, we demonstrate that existing methods are capable of producing severely biased and highly variable parameter estimates and that our novel BH-EM method significantly outperforms them in all studied circumstances. We further illustrate the performance on network flow (NetFlow) data between devices in a real large-scale computer network, to characterize triggering behavior. These results highlight the importance of correct handling of binned data. Supplementary materials for this article are available online.

MODELS AND TOOLS OF THE AUTOMATED SYSTEM FOR COMPUTER NETWORK TRAFFIC INVESTIGATION USING BERKELEY PACKET FILTER

An approach for automating the monitoring and analysis of incoming network traffic in large-scale computer networks is proposed in the paper. The authors suggest using the Linux Berkeley Packet Filter tool to automate traffic analysis in computer networks. The software structure is developed, which includes two main parts: the service machine and the user machine, it is based on the modular principle, which allows for rapid improvement and modernization of the system. The main algorithms for software functionality are built, namely: the algorithm for processing network packets using the Berkeley Packet Filter tool, and the algorithm of the user-space program for loading the Berkeley Packet Filter program to kernel space and setting up communication with it. A study model of program functioning dynamics based on the Petri net theory has been developed. As a result of the application of models based on the Petri net in the software development process, the system works correctly, all states are accessible, and there are no dead ends. A simulation model of the application of the Berkeley Packet Filter tool for the automation of computer network traffic analysis was designed, and the script was created for testing the developed software system. Implemented Python script generates a flow of network packets with random values in the sender IP address, receiver port number, and protocol. These packets, created by the IP address spoofing methodology, later are sent to the service machine's network interface. The developed computer network traffic monitoring software, that uses the Berkeley Packet Filter tool and is implemented in C, C++, and Python programming languages, provides collecting and processing of computer network traffic data. The output of the analysed results is displayed in a user-friendly form. The development and testing of the created software were carried out on the operating system Arch Linux version 5.10.3, which was previously installed on a virtual machine. The results of traffic testing in computer networks in different modes of normal operation and during DDoS attacks are given. In particular, an example of sidebar output with network traffic statistics over a long period, an example of output with network traffic spike parameters, and an example of a warning message, that the sidebar dashboard will show, are presented.

Network Performance Optimization using Dynamic Enhanced Interior Routing Protocols Gateway Routing Protocol for IPv6 (EIGRPv6) and IPv6 Access Control List

Internet network needs are increasingly rapid in supporting current business needs. This goes hand in hand with requests for IP Address allocations for companies to measure information and business needs. The increasing demand for current IP Address allocation, the fewer the number of IP Addresses available. To fulfill the demand from the use of IP Address, Internet Protocol version 6 (IPv6) or internet protocol was adopted as the next-generation Internet technology. Exchange of IP Address in computer network cannot be separated from the existence of routing protocol. Enhanced Interior Gateway Routing Protocol (EIGRP) also developed with the use of IPv6 to EIGRP for IPv6 (EIGRPv6). The EIGRPv6 routing concept is designed so well that it is capable of handling table routing that is very large and more efficient than the previous EIGRP. In large-scale computer networks must use good security. One method of network security is the Access Control List (ACL) capable of supporting the performance of IPv6. The application of ACL in this study is able to filter the access of network users by using IPv6 Extended ACL. Based on the results of EIGRPv6 network testing, the average packet delivery time is 7.6ms. And the application of IPv6 ACL is able to limit access to Web Servers, ICMP and FTP Servers properly.

Anomaly detection by discovering bipartite structure on complex networks

Anomaly detection is a classic problem on complex networks. An anomaly detection method based on network projection is proposed in this study on networks with fundamental bipartite connection relationships and repeated interactions, such as the Internet and computer networks. First of all, two network partition algorithms are advanced to discover the bipartite structure of the network. Then, put forward a similarity metric based on the cosine similarity function to construct the projection network. Finally, the metrics of vertices in the projection network are used as input of the one-class SVM algorithm to detect anomalies. Experiments on simulation datasets and real datasets illustrate that our method achieves higher precision in identifying anomalous addresses than traditional approaches for large-scale Internet and computer networks with the bipartite structure.

On the Memory Requirement of Hop-by-Hop Routing: Tight Bounds and Optimal Address Spaces

Routing in large-scale computer networks today is built on hop-by-hop routing: packet headers specify the destination address and routers use internal forwarding tables to map addresses to next-hop ports. In this paper we take a new look at the scalability of this paradigm. We define a new model that reduces forwarding tables to sequential strings, which then lend themselves readily to an information-theoretical analysis. Contrary to previous work, our analysis is not of worst-case nature, but gives verifiable and realizable memory requirement characterizations even when subjected to concrete topologies and routing policies. We formulate the optimal address space design problem as the task to set node addresses in order to minimize certain network-wide entropy-related measures. We derive tight space bounds for many well-known graph families and we propose a simple heuristic to find optimal address spaces for general graphs. Our evaluations suggest that in structured graphs, including most practically important network topologies, significant memory savings can be attained by forwarding table compression over our optimized address spaces. According to our knowledge, our work is the first to bridge the gap between computer network scalability and information-theory.

Material-integrated cluster computing in self-adaptive robotic materials using mobile multi-agent systems

Recent trends like internet-of-things (IoT) and internet-of-everything (IoE) require new distributed computing and communication approaches as size of interconnected devices moves from a cm $$^{\text {3}}$$ - to the sub-mm $$^{\text {3}}$$ -scale. Technological advance behind size reduction will facilitate integration of networked computing on material rather than structural level, requiring algorithmic and architectural scaling towards distributed computing. Associated challenges are linked to use of low reliability, large scale computer networks operating on low to very low resources in robotic materials capable of performing cluster computing on micro-scale. Networks of this type need superior robustness to cope with harsh conditions of operation. These can be provided by self-organization and—adaptivity. On macro scale, robotic materials afford unified distributed data processing models to allow their connection to smart environments like IoT/IoE. The present study addresses these challenges by applying mobile Multi-agent systems (MAS) and an advanced JavaScript agent processing platform (JAM), realizing self-adaptivity as feature of both data processing and the mechanical system itself. The MAS’ task is to solve a distributed optimization problem using a mechanically adaptive robotic material in which stiffness is increased via minimization of elastic energy. A practical realization of this example necessitates environmental interaction and perception, demonstrated here via a reference architecture employing a decentralized approach to control local property change in service based on identification of the loading situation. In robotic materials, such capabilities can support actuation and/or lightweight design, and thus sustainability.

Analytical modelling and optimization analysis of large-scale communication systems and networks with repairmen policy

The large-scale communication systems and computer networks provide flexible, efficient, and highly available services to their users. However, the practical large-scale systems result in unpredictable, fault-tolerant, often detrimental outcomes. This leads to developing and designing analytical models to understand and predict of complex system behaviour in order to ensure availability of large-scale systems. In this paper, analytical modelling and optimization analysis are presented for large-scale systems. The key contribution of this paper is twofold. First, a generic approximate solution approach is adapted and developed for performability modelling which considers performance and availability issues of large number of nodes with multi-repairmen. The analytical model and solution presented here are capable of considering large number of nodes up to thousands and able to incorporate availability issues of the system. Second and foremost, the relationship between the number of nodes and the number of repairmen is presented with an optimization analysis for large-scale systems. In order to show the efficacy and the accuracy of the proposed approach, the results obtained from the analytical model is validated with the results obtained from the simulations.

Fault-tolerant cycles embedding in folded hypercubes

The generalized conditional fault-tolerant embedding is investigated, in which the n-dimensional folded hypercube networks (denoted by FQ n ) acts as the host graph, and the longest fault-free cycle represents the guest graph. Under the conditions looser than that of previous works, it is shown that FQ n has a cycle with length at least 2 n -2|F v | when the number of faulty vertices and non-critical edges is at most 2n - 4; where |F v | is the number of faulty vertices. It provides further theoretical evidence for the fact that FQ n has excellent node-fault-tolerance and edge-fault-tolerance when used as a topology of large scale computer networks.

The Role of Aesthetics and Perception in Raising Situation Awareness: Lessons from SpringRain

In the face of increased cyber risks, we present our iterative design process and the resulting principles of SpringRain, an information visualization (infoVis) display design concept for large screens in network control rooms (NOCs). It aims to raise team situation awareness by visualizing large-scale multidimensional computer network data sets as a live “rainfall.” We used aesthetic principles and theories of perception to prototype this ambient, yet data-dense, visualization. By applying multiple data dimensions to different properties of a line segment, such as length, motion, and color, the two-dimensional visualization offers analytical affordances (i.e., graphic qualities that make it clear how the display should be “read”) that can be processed pre-attentively (i.e., under 200 ms). This grants that time-sensitive anomalies in the network can be noticed, processed, and addressed in a timely manner. Our design approach was driven by theories rather than existing design works in the hope of encouraging more user-centered theories-driven infoVis designs that are better suited to user needs and requirements. Expert reviewers’ feedback from the VAST 2013 Challenge committee confirmed the novelty of the design and pointed to opportunities for improvements. We discuss three redesign attempts to address some of the identified limitations.

The Application Research in the Coal Mine on Chirp-UWB Wireless Communication Technology

With the rapid development of wireless network technology, according to the actual need, the computer network group network type and each node has different function, so that the computer network system has node complexity, structure complexity and various other complexities. For these reasons, the large scale computer network can be treated and researched abstractly as specific Chirp-UWB network. Therefore, to keep up with the trend of technology progress by means of computer wireless technology, aided design of wireless communication system, it is quite necessary for the implementation of computer management of production safety.

Authentication graphs: Analyzing user behavior within an enterprise network

User authentication over the network builds a foundation of trust within large-scale computer networks. The collection of this network authentication activity provides valuable insight into user behavior within an enterprise network. Representing this authentication data as a set of user-specific graphs and graph features, including time-constrained attributes, enables novel and comprehensive analysis opportunities. We show graph-based approaches to user classification and intrusion detection with practical results. We also show a method for assessing network authentication trust risk and cyber attack mitigation within an enterprise network using bipartite authentication graphs. We demonstrate the value of these graph-based approaches on a real-world authentication data set collected from an enterprise network.

Multicast and customized deployment of large-scale operating systems

With the recent paradigm shift of cloud computing, deployment of operating systems (OSs) onto a large-scale computer network is becoming necessary. Note that there are usually numerous nodes with various functions in a cloud computing system. Thus, it is usually required to deploy different operating systems onto different nodes. In such a customized setting, conventional techniques of using unicast deployment to distribute a massive cloud OS onto thousands of nodes is time consuming and bandwidth-intensive. In this work, we thus propose a multicast deployment approach so as to significantly improve deployment efficiency. Furthermore, our multicast deployment approach can leverage existing configurations of the unicast counterpart. Specifically, the advantageous features of the proposed approach include the support of a reliable multicast protocol, a heterogeneous infrastructure, and cloud hypervisor environments. To evaluate the feasibility of the proposed approach in practical applications, CentOS and Ubuntu are used when implementing our deployment approach on several tens of nodes. Empirical studies show that both the required time for the entire distribution process, i.e., from starting delivery until the OS is ready, and the network bandwidth consumption are significantly reduced as compared with conventional unicast approaches. Consequently, less effort is required on monitoring and maintenance for system administrators.

BANKSAFE: Visual analytics for big data in large-scale computer networks

The enormous growth of data in the last decades led to a wide variety of different database technologies. Nowadays, we are capable of storing vast amounts of structured and unstructured data. To address the challenge of exploring and making sense out of big data using visual analytics, the tight integration of such backend services is needed. In this article, we introduce BANKSAFE, which was built for the VAST Challenge 2012 and won the outstanding comprehensive submission award. BANKSAFE is based on modern database technologies and is capable of visually analyzing vast amounts of monitoring data and security-related datasets of large-scale computer networks. To better describe and demonstrate the visualizations, we utilize the Visual Analytics Science and Technology (VAST) Challenge 2012 as case study. Additionally, we discuss lessons learned during the design and development of BANKSAFE, which are also applicable to other visual analytics applications for big data.

On the 2-MRS Problem in a Tree with Unreliable Edges

This paper extends the well-known most reliable source (1-MRS) problem in unreliable graphs to the 2-most reliable source (2-MRS) problem. Two kinds of reachable probability models of node pair in unreliable graphs are considered, that is, the superior probability and united probability. The 2-MRS problem aims to find a node pair in the graph from which the expected number of reachable nodes or the minimum reachability is maximized. It has many important applications in large-scale unreliable computer or communication networks. The #P-hardness of the 2-MRS problem in general graphs follows directly from that of the 1-MRS problem. This paper deals with four models of the 2-MRS problem in unreliable trees where every edge has an independent working probability and devises a cubic-time and quadratic-space dynamic programming algorithm, respectively, for each model.

An epistemic event-based correlation approach for managing pervasive networks

Existing pervasive applications are based on time series data that possess the form of time-ordered series of events. Such applications also embody the need to handle large volumes of unexpected events, often modified on-the-fly, containing conflicting information, and dealing with rapidly changing contexts while producing results with low latency. Correlating events across contextual dimensions holds the key to expanding the capabilities and improving the performance of these applications. In this paper we analyze complex-event semantic correlation that examines epistemic uncertainty in computer networks by using Dempster–Shafer theory to support a high-volume, event-based, in-network and non-deterministic pervasive network management. We consider imprecision and uncertainty when an event is detected and associate a belief parameter with the semantics and the detection of composite events. The approach taps into in-network processing capabilities of pervasive computer networks and can withstand missing or conflicting information gathered from multiple participating entities. In the end, we establish that a lightweight, distributed, large-volume, event-based technique which exploits epistemic uncertainty to correlate events along contextual dimensions provides a successful technique for enabling management of large-scale and pervasive contemporary and future computer networks. Copyright © 2011 John Wiley & Sons, Ltd.

GetCM: A Vision of Cloud Manufacturing

Surviving in an increasing globalization, distribution and flexibility environment, modern manufacturing requires an extremely flexible, self-adaptive foundation capable of dynamic provisioning, coordinating and using infinite manufacturing resources available on demand over large-scale computer networks. In contrast to the conventional networked manufacturing approach, the cloud manufacturing vision (GetCM) introduced in this paper promises elasticity, flexibility and adaptability through the on-demand provisioning of manufacturing resources as a utility by reflecting the basic principles of cloud computing. The discussion is made from technological, functional, economic aspects to provide evidence of the benefits from GetCM in the context of networked manufacturing resource access, provision, sharing and coordination. A primary architecture for GetCM is introduced based on the analysis of key criteria in realizing the vision of a function for cloud manufacturing. Focuses of this paper are placed on the vision and the outline of GetCM architecture.

Bandwidth Modeling and Estimation in Peer to Peer Networks

Recent studies have shown that the majority of today's internet traffic is related to Peer to Peer (P2P) traffic. The study of bandwidth in P2P networks is very important. Because it helps us in more efficient capacity planning and QoS provisioning when we would like to design a large scale computer networks. In this paper motivated by the behavior of peers (sources or seeds) that is modeled by Ornstein Uhlenbeck (OU) process, we propose a model for bandwidth in P2P networks. This model is represented with a stochastic integral. We also model the bandwidth when we have multiple downloads or uploads. The autocovariance structure of bandwidth in either case is studied and the statistical parameters such as mean, variance and autocovariance are obtained. We then study the queue length behavior of the bandwidth model. The methods for generating synthetic bandwidth process and estimation of the bandwidth parameters using maximum likehood estimation are presented.

Multidecision Quickest Change-Point Detection: Previous Achievements and Open Problems

The following multidecision quickest detection problem, which is of importance for a variety of applications, is considered. There are N populations that are either statistically identical or where the change occurs in one of them at an unknown point in time. Alternatively, there may be N “isolated” points/hypotheses associated with a change. It is necessary to detect the change in distribution as soon as possible and indicate which population is “corrupted” or which hypothesis is true after a change occurs. Both the false alarm rate and misidentification rate should be controlled by given (usually low) levels. We discuss performance of natural multihypothesis/multipopulation generalizations of the Page and Shiryaev-Roberts procedures, including certain asymptotically optimal properties of these tests when both the false alarm and the misidentification rates are low. Specifically, we show that under certain conditions the proposed multihypothesis detection-identification procedures asymptotically minimize the trade-off between any positive moment of the detection lag and the false alarm/misclassification rates in the worst-case scenario. At the same time, the corresponding sequential detection-identification procedures are computationally simple and can be easily implemented online in a variety of applications such as rapid detection of intrusions in large-scale distributed computer networks, target detection in cluttered environment, and detection of terrorist' malicious activity. Limitations of the existing and proposed solutions to this challenging problem are also discussed.

Preface: New Challenges in Large-Scale Computer Systems and Network Modeling and Simulation

Preface: New Challenges in Large-Scale Computer Systems and Network Modeling and Simulation