Internet Data Analysis Research Articles

Modalidad MOOC para educación media básica: enseñanzas de una experiencia

El artículo analiza la eficiencia terminal y los diferentes escenarios de tecnología, conectividad y aprendizaje que caracterizaron la experiencia del curso SCRACHT SM4T, organizado por el Plan Ceibal de Uruguay, en convenio con una universidad privada. La metodología aplicada es de tipo descriptiva transeccional, y se basa en el análisis de variables, registros y datos de Internet de los estudiantes. El estudio mostró que es posible implementar nuevas ofertas de cursos MOOC en escenarios de menor o mayor nivel de formalización, mediante diferentes estrategias de apoyo tutorial docente. Uno de los hallazgos principales de la investigación es que la tasa de finalización es superior a las reportadas por la literatura internacional, y en la mayoría de los estudios sobre MOOC. Un segundo aporte es la identificación y análisis de tres tipos de escenarios formativos: adaptativo individual, grupal con apoyo de tutor, y masivo mediante inscripción desde un centro educativo formal.

Manufaktura in Łódź, Poland: An example of a festival marketplace

ABSTRACTThe main aim of the article is to describe the phenomenon of the festival marketplace using the example of Manufaktura in Łódź, which is a post-socialist Polish city. The article presents the concept, the development of large shopping centres, and the main socio-economic and spatial urban transformations that accompanied the creation of Manufaktura. The research methods included analysis of Internet and statistical data, participant observation, one in-depth interview, and a large number of semi-structured interviews. The results of the study confirm that Manufaktura – a large arts centre, shopping mall, and leisure complex built in the revitalized part of a former textile factory – represents most of the festival marketplace features. Additionally, revitalization reflects urban transformation processes in post-socialist cities, such as functional transformation, deindustrialization, internationalization of trade, privatization, or changes in consumer behaviour. The author concludes that Manufaktura is now an important part of the revitalized urban geographical space in Łódź. It performs essential urban functions, not only commercial but also cultural and tourism-related.

Teaching Network Security With IP Darkspace Data

This paper presents a network security laboratory project for teaching network traffic anomaly detection methods to electrical engineering students. The project design follows a research-oriented teaching principle, enabling students to make their own discoveries in real network traffic, using data captured from a large IP darkspace monitor operated at the University of California, San Diego (UCSD). Although darkspace traffic does not include bidirectional conversations (only attempts to initiate them), it contains traffic related to or actually perpetrating a variety of network attacks originating from millions of Internet addresses around the world. This breadth of coverage makes this darkspace data an excellent choice for a hands-on study of Internet attack detection techniques. In addition, darkspace data is less privacy-critical than other network traces, because it contains only unwanted network traffic and no legitimate communication. In the lab exercises presented, students learn about network security challenges, search for suspicious anomalies in network traffic, and gain experience in presenting and interpreting their own findings. They acquire not only security-specific technical skills but also general knowledge in statistical data analysis and data mining techniques. They are also encouraged to discover new phenomena in the data, which helps to ignite their general interest in science and engineering research. The Vienna University of Technology, Austria, first implemented this laboratory during the summer semester 2014, with a class of 41 students. With the help of the Center for Applied Internet Data Analysis (CAIDA) at UCSD, all exercises and IP darkspace data are publicly available.

Technology for collecting and analyzing relational data

Purpose of the study. The scientific and educational organizations use traditionally e-mail with Microsoft Excel spreadsheets and Microsoft Word documents for operational data collection. The disadvantages of this approach include the lack of control of the correctness of the data input, the complexity of processing the information received due to non-relational data model, etc. There are online services that enable to organize the collection of data in a relational form. The disadvantages of these systems are: the absence of thesaurus support; a limited set of elements of data input control; the limited control the operation of the input form; most of the systems is shareware, etc. Thus, it is required the development of Internet data collection and analysis technology, which should allow to identify quickly model the data collected and automatically implement data collection in accordance with this model. Materials and methods. The article describes the technology developed and tested for operational data collection and analysis using Faramant system. System operation Faramant is based on a model document, which includes three components: description of the data structure; visualization; logic of form work. All stages of the technology are performed by the user using the browser. The main stage of the proposed technology is the definition of the data model as a set of relational tables. To create a table within the system it’s required to determine the name and a list of fields. For each field, you must specify its name and use the control to the data input and logic of his work. Controls are used to organize the correct input data depending on the data type. Based on a model system Faramant automatically creates a filling form, using which users can enter information. To change the form visualization, you can use the form template. The data can be viewed page by page in a table. For table rows, you can apply different filters. To summarize the information there is a mechanism of data grouping, which provides general data of the number of entries, maximum, minimum, average values for different groups of records. Results. This technology has been tested in the monitoring requirements of the services of additional professional education and the definition of the educational needs of teachers and executives of educational organizations of the Irkutsk region. The survey has involved 2,780 respondents in 36 municipalities. Creating the data model took several hours. The survey was conducted during the month. Conclusion. The proposed technology allows a short time to collect the information in relational form, and then analyze it without the need for programming with flexible assignment of the operating logic for form.

Characterising heterogeneity in vulnerable hosts on worm propagation

Worm propagation models are important for understanding worm dynamics and designing effective and efficient detection and defence systems. The existing models, however, ignore the heterogeneity in vulnerable hosts and assume that the worm-scanning rate is the same for all infected hosts. In this work, we analytically and empirically study the impact of heterogeneity of vulnerable hosts on worm propagation. Specifically, we first apply the Jensen's inequality to show that the heterogeneity in vulnerable hosts indeed hinders the speed of worm propagation. Next, we propose a novel model to predict and characterise worm dynamics among heterogeneous vulnerable hosts. Finally, applying the real trace from Center for Applied Internet Data Analysis CAIDA to simulate the propagation of a Witty worm in the Internet, we verify our analytical results and demonstrate that our proposed model can accurately predict the spread of worms among heterogeneous vulnerable hosts.

A Hierarchical Internet Path Available Bandwidth Inference Mechanism

Distributed applications such as cloud computing and CDN need the Internet state information to optimize their performance. Although the present network coordinate systems are effective in latency inference, they cannot infer performance metric such as available bandwidth. Therefore, this paper attempts to use a novel mechanism to concentrate on how to infer the Internet path available bandwidth. Based on theoretical and Internet measurement data analysis result, we discover that Internet path available bandwidth has ultra metric space feature, based on which we propose an available bandwidth inference mechanism called PathGuru. In PathGuru, each node uses the partial measurement data to and from landmarks to obtain the outgoing and the incoming bandwidth vectors, and then the available bandwidth between any two nodes can be inferred according to the two vectors as well as the ultra metric space constraint. Experimental results on the Internet measurement dataset indicate that PathGuru has reasonable inference accuracy and can also deal with the available bandwidth symmetry violation problem.

The impact of the Internet on global industry: New evidence of Internet measurement

This paper presents a new way to measure the diffusion of the Internet, using a panel of 10 countries. Different from the previous literature, we use city-level daily databases, downloaded from Cooperative Association for Internet Data Analysis (CAIDA), to construct weekly data across countries. The determinants of Internet cross-traffic traveling are studied using several attributes which offer new opportunities to define network construction and information technology. With the Swiss KOF Globalization Index, this data can be used to reevaluate financial services and business marketing, political interaction and social emergence, and globalization. The major findings are that a significant and positive relationship exists between Internet distance and the different globalization indexes: economic and financial globalization, political globalization, and social globalization. The Internet flows ahead of globalization. The dynamic panel causality analysis demonstrates further empirically the causality of the Internet diffusion on the different indexes of globalization.

Inefficiency of IDS Static Anomaly Detectors in Real-World Networks

A wide range of IDS implementations with anomaly detection modules have been deployed. In general, those modules depend on intrusion knowledge databases, such as Knowledge Discovery Dataset (KDD99), Center for Applied Internet Data Analysis (CAIDA) or Community Resource for Archiving Wireless Data at Dartmouth (CRAWDAD), among others. Once the database is analyzed and a machine learning method is employed to generate detectors, some classes of new detectors are created. Thereafter, detectors are supposed to be deployed in real network environments in order to achieve detection with good results for false positives and detection rates. Since the traffic behavior is quite different according to the user’s network activities over available services, restrictions and applications, it is supposed that behavioral-based detectors are not well suited to all kind of networks. This paper presents the differences of detection results between some network scenarios by applying traditional detectors that were calculated with artificial neural networks. The same detector is deployed in different scenarios to measure the efficiency or inefficiency of static training detectors.

Network coding is the 5G Key Enabling Technology: effects and strategies to manage heterogeneous packet lengths

ABSTRACTThe exponential growth of the mobile devices market, not only smartphones but also tablets, laptops or wearables, poses a serious challenge for 5G communications. Random linear network coding is a promising solution for present and future networks as it has been shown to provide increased throughput, security and robustness for the transmission of data through the network. Most of the analysis and the demonstrators have focused on the study of data packets with the same size (number of bytes). This constitutes a best‐case scenario as coded packets will incur little overhead to handle such packets. However, packet lengths are quite heterogeneous in real networks, which can cause a high overhead or, alternatively, a high delay in the transmission of data packets. As we show, this can have a severe effect on a variety of applications. This paper proposes a series of mechanisms to manage heterogeneous packet lengths and analyses the induced overhead of those mechanisms using real packet length distributions provided by the Center for Applied Internet Data Analysis (CAIDA) and own measurements using video content. Our results show that an appropriate packetisation of the data packets can reduce the overhead from over 100 per cent (zero‐padding packets) to below 5 per cent for CAIDA packet distributions and that the right choice of parameters can reduce the overhead to less than 10 per cent in the case of video data. Copyright © 2014 John Wiley & Sons, Ltd.

An Intelligent DDoS Attack Detection System Using Packet Analysis and Support Vector Machine

Nowadays, many companies and/or governments require a secure system and/or an accurate intrusion detection system (IDS) to defend their network services and the user’s private information. In network security, developing an accurate detection system for distributed denial of service (DDoS) attacks is one of challenging tasks. DDoS attacks jam the network service of the target using multiple bots hijacked by crackers and send numerous packets to the target server. Servers of many companies and/or governments have been victims of the attacks. In such an attack, detecting the crackers is extremely difficult, because they only send a command by multiple bots from another network and then leave the bots quickly after command execute. The proposed strategy is to develop an intelligent detection system for DDoS attacks by detecting patterns of DDoS attack using network packet analysis and utilizing machine learning techniques to study the patterns of DDoS attacks. In this study, we analyzed large numbers of network packets provided by the Center for Applied Internet Data Analysis and implemented the detection system using a support vector machine with the radial basis function (Gaussian) kernel. The detection system is accurate in detecting DDoS attacks.

Investigation of geographic relationship between farmer's market with live birds and human infections with avian influenza A(H7N9)virus based on internet data analysis

Farmer's markets with live birds (FMLB) are key sites where human infections by influenza A virus subtype H7N9 happened. Approximately 80% cases have exposed to FMLB. This study is to investigate the geographic relationship between FMLB and human cases based on analysis of internet data of their geographic locations. Using big data from internet, we searched all FMLB in the cities where the human cases have been reported, then analyzed geographic relations, and evaluated the possibility of visits of the patients to the FMLB around them. The densities of FMLB, population and live poultries were also analyzed. Forty-two cities and 10 615 markets were included in the study. It is indicated that the number of human cases has positive correlations with the population density, the number and density of markets. Except three markets in Foshan, human cases have been reported within 5 km of 10 of 13 markets, which shows that the live bird trading is highly relevant with the distribution of cases. We identified 13 hot spots in the cities including Hangzhou, Shenzhen, et al, where clustered cases have emerged. The numbers of human cases are significantly high in cities where FMLB are detected positive for H7N9 virus. These virus positive markets usually affect the people's residence within 5km area. The number and location of FMLB in cities should be re-evaluated and re-planed for healthy city where the risk of residents infecting avian influenza virus is greatly reduced or eliminated.

Metrics for Assessing Internet Business Models and Sustainability

As the internet changes structure and behavior in ways better modeled in terms of modular composition than as an architecture based on functional links between physical layers and services, well established network metrics appear increasingly less suited to assessing economic and business strategy features. This change is described by Labovitz (Labovitz, Lekel-Johnson et al. 2010), Fransman (Fransman 2010), Clark (Clark, Lehr et al. 2011), Yoo (Yoo 2012), Frischmann (Frischmann 2012) and in our own recent work (Liebenau, Elaluf-Calderwood et al. 2012). Increasingly, measurement approaches such as those conducted by the Center for Advanced Internet Data Analysis at the University of California, San Diego [CAIDA] are reassessing what when, how and why metrics are collected. The structure of emerging business models, the value generated from certain kinds of products and services delivery, and the spillover effects that concern public policy are all masked by the typical focus on broadband speed, routing tables and traffic estimates between nodes. Along with current discussions about “quality of experience” (Kruse 2009), and new approaches to concentration (Noam 2010), we address the problem of how to assess information asymmetries (Claffy 2008), the relationships among different kinds of traffic (transit, private, peered, intra-network, etc.), flows of funds and other features that characterize the interrelationships of all major aspects of internet core businesses, whether regulated or not. In this paper we describe a view of how changes to the layered model of internet architecture has created space for new services and products and in particular the mixing of roles across previous boundaries described in part by the distinction between regulated and unregulated segments. We then consider the utility of current metrics and show how additional or alternative metrics can enhance our understanding such that we can come to a far better view of trends and practices on the internet. We show what minimum requirements there are to data that ought to be visible to all stakeholders of the internet. Through a detailed taxonomy of different sources of internet metrics (e.g. traffic through internet exchanges, charges for services, Cisco network data, Sandvine traffic statistics, etc.) we show the utility of new ways to capture and assess the important economic characteristics that will help in promoting the sustainability of the internet. We begin with a critique of a typical matrix describing web measurements associated with the physical and logical layers of the internet (e.g. from CENDI of the U.S. Government; Hodge, 2000). We assess the value of the criteria traditionally used and show how some are misleading, some inappropriate, and some useful when considered in conjunction with new criteria of assessment. We show how a careful inclusion of jurisdictional information, government fiscal policies and regulation, costing estimates, maintenance, network qualities and other evidence from political economy can provide a superior metric that will provide the basis for better economic analyses and in judging business models and sustainability.

Analysis of Interlayer Connection Catastrophe Characteristics in Internet AS level Topology

Based on CAIDA’s (cooperative association for Internet data analysis) statistics at AS-level collected from April 2009 to April 2010, it is found that network size, degree correlations and clustering have changed significantly three times. According to the definition of k-core, the Internet topology is divided into different levels. In this paper, by analyzing the changing connection on each level at the three time spots, we found that the connection difference between the highest and the lowest level varies cyclically. Before Internet has significant change, the connection difference between the highest and the lowest shell changes acutely. At that time point it is the later stage of the fluctuation period which is the accumulation of small fluctuation. The concept of catastrophe coefficient is proposed for quantifying the probability of Internet catastrophe so that the catastrophe coefficient of Internet could be calculated on real time, and the catastrophe time spots could be predicted. Using real data for verification, it turns out that catastrophe coefficient can accurately predict the giant fluctuations occurrence in the macroscopic structure. With the proposal of the concept of catastrophe coefficient, the research can play an important guiding role in understanding the actual network internal development and changes, analyzing the cause of Internet catastrophe, preventing devastating catastrophe, planning and further re-designing Ipv6. DOI: http://dx.doi.org/10.11591/telkomnika.v11i2.1978

A Fluid-Based Approach for Modeling Network Activities

Network traffic traces provide valuable information for researchers to study behaviors of normal and malicious network activities. Although traffic traces are enough to reveal packet-level and connection-level details of most network activities, identifying specific malicious network activities is still a huge challenge: many malicious network activities are able to hide themselves behind normal activities with forged packet and connection information. In practice, mechanisms that are able to effectively extract malicious network activities from raw traffic traces are emerging and will benefit network security and other related communities as well. In this paper, a fluid-based approach for modeling simulated normal and malicious flooding-based denial of service network activities is developed. To approach this goal, several raw traffic traces gathered by the Cooperative Association for Internet Data Analysis (CADIA) are analyzed and investigated.

Learning electoral geography? Party campaigning, constituency marginality and voting at the 2010 British general election

Multi‐party single‐member constituency electoral systems such as Great Britain’s are characterised by the political parties carefully targeting their local campaigns to maximise the number of seats won. They focus their activities on constituencies where either victory is likely or defeat is possible, which involves ensuring that residents are aware of the local context and the party’s electoral prospects there. But how do voters assess a party’s chances of victory locally; how aware are they of the local situation and to what extent does party campaigning change this? Analysis of British Election Study internet panel survey data shows that although many voters had an accurate assessment of each party’s chance of victory locally in 2010 many either over‐ or under‐estimated its chances relative to the actual situation. Furthermore a substantial proportion changed their assessments as the election campaign progressed. Those contacted by a party during the various stages of the campaign tended to assess its chances of local victory as higher than those not contacted, especially if they also thought it had a good chance of winning the contest nationally, and such individuals were more likely to vote for the party than those not contacted.

Understanding and Elevating Web Practice Through Web Analytics Final

Web analytics is the measurement, collection, analysis and summarizing of internet data for purposes of understanding and optimizing web usage. To check the web analytics the website IJCT is considered. This paper is the first step to analyze the availability and performability of the site. Availability depicts that for how much time the system is up i.e. check the utilization of the system on which that site works. Performability is the method to analyze each and every module separately and clearly in an efficient manner. Availability and performability collectively check the reliability of the system to ensure the performance measurement of that system. Web analytics is not just a tool for measuring and depicting website traffic but can be used as a tool for business research and market research, as well as a means to check and improve upon the effectiveness and utilization of a web site .The tool used for the web analytics for this site is AWStats. AWStats is an open source Web analytics reporting tool, which used for analyzing the amount of data from Internet services such as web, streaming media, mail and FTP servers .This tool check the performance of the IJCT separately by dividing one month module to days in month, week in month ,hours in month ,contribution of various countries, traffic generated etc. This paper deal with the securitization of IJCT for January month to indicate how many events and functionality occurs during surfing on this journal by various users on the Web. This critique helps us to identify the throughput of IJCT.  Here it is clear that users do not actually read: instead, they download and upload the data. A study of different online journals defines that this website scored massive response instead of the short time span. It completely and efficiently elaborate the usage of the IJCT by stating user visited, number of visits, number of hits ,bandwidth and many more.

Studying the Effects of Internet Exchange Points on Internet Topology

The recent uncovering of a high number of peering links at Internet eXchange Point (IXP) locations across the world has made these exchange switches a critical component of the Internet Autonomous System (AS) level ecosystem. Studies concentrating on the Internet topology evolution have surmised that numerous links hidden at these exchange points hold the key towards solving the missing links problem in studying the evolution of the AS-level topology of the Internet. In this work, we study the e?ect of this?set of hitherto unseen peering links on the visible Internet topology. Starting with a set of measurements determining the growth of IXPs in the inter-domain routing architecture of the Internet and continuing with a more advanced graph based metric analysis of available Internet topology data, we conclude that IXP links follow power law increase characteristics while exhibiting definitive clustering characteristics. Moreover, these additional links a?ect the joint degree distributions of nodes with higher degrees while?leaving most other types of nodes unchanged. We conclude that the currently inferred AS-level maps of the Internet demonstrate considerable variations with the incorporation of these new links and could eventually lead to a remodeling of our understanding of Internet topology evolution.

Passive IP traceback

IP traceback can be used to find the origin of anonymous traffic; however, Internet-scale IP traceback systems have not been deployed due to a need for cooperation between Internet Service Providers (ISPs). This article presents an Internet-scale Passive IP Trackback (PIT) mechanism that does not require ISP deployment. PIT analyzes the ICMP messages that may scattered to a network telescope as spoofed packets travel from attacker to victim. An Internet route model is then used to help re-construct the attack path. Applying this mechanism to data collected by Cooperative Association for Internet Data Analysis (CAIDA), we found PIT can construct a trace tree from at least one intermediate router in 55.4% the fiercest packet spoofing attacks, and can construct a tree from at least 10 routers in 23.4% of attacks. This initial result shows PIT is a promising mechanism.

Characterizing and modeling the Internet Router-level topology – The hierarchical features and HIR model

In this paper, we seek to understand the properties related with the hierarchical structure of the Internet Router-level topology. We first analyzed the hierarchical features of the Internet Router-level topology with the actual topology measuring data authorized by CAIDA (The Cooperative Association for Internet Data Analysis) Skitter project. Then we analyzed the distribution of the nodes in each shell, the distribution of the edges within the shell and to the higher shells and induced the computing functions. We proposed a new framework for modeling the Internet Router-level topology with the conclusion we have drawn and we implemented the model, HIR model. HIR model was based on the k-core decomposition in order to reveal the hierarchical of the network. The analysis shows that HIR model can reproduce Internet on most properties and has got the hierarchies of network close to that of Internet. The experiments show that HIR model can fully reflect the hierarchy characteristics of the Internet Router-level topology while preserving the power-law distribution on degree.

Chaotic Time Series Analysis

Chaotic dynamical systems are ubiquitous in nature and most of them does not have an explicit dynamical equation and can be only understood through the available time series. We here briefly review the basic concepts of time series and its analytic tools, such as dimension, Lyapunov exponent, Hilbert transform, and attractor reconstruction. Then we discuss its applications in a few fields such as the construction of differential equations, identification of synchronization and coupling direction, coherence resonance, and traffic data analysis in Internet.