Scalable Big Data Research Articles

Securing Information Exchange in VANETs by Using Pairing-Based Cryptography

Vehicular Ad Hoc Networks are mainly implemented to enable the interchange of huge amount of information among vehicles and between vehicles and control entities such as road side units or base stations, providing support for a comfortable and safe driving experience. However, due to the recent proliferation of cybersecurity threats, securing such a critical exchange of information becomes a fundamental prerequisite. In this paper, we propose a novel security scheme based on bilinear pairing-based cryptography to improve the security of the information exchanged in VANETs. Such scheme relies on the Elliptic Curve Discrete Logarithm Problem to provide anonymity and robust security features, and on Message Authentication Codes for verifying the vehicles’ identities. The proposed solution is able to achieve mutual authentication between involved entities and prevent impersonation, replay and insider attacks, at the expense of minimum overhead so that also big-data scale communications can be safely supported in the VANET environment.

Scaling Up Integrated Structural and Content-Based Network Analysis

Structural analysis of social networks can provide important insights into the clusters and important nodes. However, it is silent on the content-based reasons for importance or commonality. This additional level of insight requires sampling content from nodes and processing it to distill new insights. That is done effectively by human analysts, but as networks grow into Big Data scale, human analysis is not possible. This raises the question of whether automated techniques can mimic the same results humans find. In this paper, we demonstrate how topic modeling can be applied, filtered, and adapted to produce easy-to-understand keywords that represent important clusters in a network. Those keywords reflect the insights achieved by human analysts doing a manual content-based analysis of the network features. While humans should never be removed from the analysis process, this work shows how automated techniques can be integrated to scale humans’ ability to gain insights in large networks.

Bigdata Driven Cloud Security: A Survey

Cloud Computing (CC) is a fast-growing technology to perform massive-scale and complex computing. It eliminates the need to maintain expensive computing hardware, dedicated space, and software. Recently, it has been observed that massive growth in the scale of data or big data generated through cloud computing. CC consists of a front-end, includes the users’ computers and software required to access the cloud network, and back-end consists of various computers, servers and database systems that create the cloud. In SaaS (Software as-a-Service – end users to utilize outsourced software), PaaS (Platform as-a-Service-platform is provided) and IaaS (Infrastructure as-a-Service-physical environment is outsourced), and DaaS (Database as-a-Service-data can be housed within a cloud), where leading / traditional cloud ecosystem delivers the cloud services become a powerful and popular architecture. Many challenges and issues are in security or threats, most vital barrier for cloud computing environment. The main barrier to the adoption of CC in health care relates to Data security. When placing and transmitting data using public networks, cyber attacks in any form are anticipated in CC. Hence, cloud service users need to understand the risk of data breaches and adoption of service delivery model during deployment. This survey deeply covers the CC security issues (covering Data Security in Health care) so as to researchers can develop the robust security application models using Big Data (BD) on CC (can be created / deployed easily). Since, BD evaluation is driven by fast-growing cloud-based applications developed using virtualized technologies. In this purview, MapReduce [12] is a good example of big data processing in a cloud environment, and a model for Cloud providers.

Size Matters to Lesbians, Too: Queer Feminist Interventions into the Scale of Big Data

How can we recognize those whose lives and data become attached to the far-from-groundbreaking framework of “small data”? Specifically, how can marginalized people who do not have the resources to produce, self-categorize, analyze, or store “big data” claim their place in the big data debates? I examine the place of lesbians and queer women in the big data debates through the Lesbian Herstory Archive's not “big” enough lesbian, gay, bisexual, trans, and queer (LGBTQ) organizing history data set—perhaps the largest data set known to exist on LGBTQ activist history—as one such alternative. In a contribution to critical data studies, I take a queer feminist approach to the scale of big data by reading for the imbricated scales and situated knowledge of data.

MarDRe: efficient MapReduce-based removal of duplicate DNA reads in the cloud.

This article presents MarDRe, a de novo cloud-ready duplicate and near-duplicate removal tool that can process single- and paired-end reads from FASTQ/FASTA datasets. MarDRe takes advantage of the widely adopted MapReduce programming model to fully exploit Big Data technologies on cloud-based infrastructures. Written in Java to maximize cross-platform compatibility, MarDRe is built upon the open-source Apache Hadoop project, the most popular distributed computing framework for scalable Big Data processing. On a 16-node cluster deployed on the Amazon EC2 cloud platform, MarDRe is up to 8.52 times faster than a representative state-of-the-art tool. Source code in Java and Hadoop as well as a user's guide are freely available under the GNU GPLv3 license at http://mardre.des.udc.es . rreye@udc.es.

CLOUD-BASED DATA ANALYTICS FRAMEWORK FOR MOBILE APP EVENT ANALYSIS

Mobile analytics studies the behavior of end users of mobile applications and the mobile application itself. These mobile applications, being an important part of the various businesses products, need to be monitored and the usage patterns are to be analyzed. The data collected from these apps can help to drive important business strategies by identifying the usage patterns. Enriching the data with information available from other sources, like sales/service information, provides holistic view about the solution. Thus, here we aim at exploring some set of tools that give capabilities as event trailing with higher extraction of its linguistics. If the application is used worldwide, the data generated out of it is Big Data, which traditional systems cannot handle. We therefore propose a special framework for efficient data collection, storage and processing at Big Data scale on cloud platform.

Television by the numbers: The challenges of audience measurement in the age of Big Data

This article examines recent innovations in how television audiences are measured, paying particular attention to the industry's growing efforts to utilize the large bodies of data generated through social media platforms – a paradigm of research known as Big Data. Although Big Data is considered by many in the television industry as a more veracious model of audience research, this essay uses Boyd and Crawford's (2011) `Six Provocations of Big Data' to problematize and interrogate this prevailing industrial consensus. In doing so, this article explores both the affordances and the limitations of this emerging research paradigm – the latter having largely been ignored by those in the industry – and considers the consequences of these developments for the production culture of television more broadly. Although the full impact of the television industry's adoption of Big Data remains unclear, this article traces some preliminary connections between the introduction of these new measurement practices and the production culture of contemporary television. First, I demonstrate how the design of Big Data privileges real-time analysis, which, in turn, encourages increased investment in ‘live’ and/or ‘event’ television. Second, I argue that despite its potential to produce real-time insights, the scale of Big Data actually limits its utility in the context of the creative industries. Third, building on this discussion of the debatable value and applicability of Big Data, I describe how the introduction of social media metrics is further contributing to a ‘data divide’ in which access to these new information data sets is highly uneven, generally favouring institutions over individuals. Taken together, these three different but overlapping developments provide evidence that the introduction of Big Data is already having a notable effect on the television industry in a number of interesting and unexpected ways.

A Maturity Analysis of Big Data Technologies

Big Data OverviewDriven by the need to generate business value, the enterprise has started to adopt Big Data as a solution, migrating from the classical databases and data stores which lack the flexibility and are not optimized enough [1].The changes in the environment make big data analytics attractive to all types of organizations, while the market conditions make it practical. The combination of simplified models for development, commoditization, a wider palette of data management tools, and low-cost utility computing has effectively lowered the barrier to entry. [2]. The concept addresses large volumes of complex data, rapid growing data sets that may come from different autonomous sources.In recent approaches, Big Data is characterized by principles known as the 4VVolume, Variety, Velocity and Veracity [3]. There are opinions about accepting other principles as Big Data characteristics, such as Value.Each day more businesses realize that Big Data is relevant as the applications generate large volumes of data generated automatically, from different data sources, centralized or autonomous. As traditional databases hit limitations when the need of analyzing this data, dedicated solutions must be considered.Important BigData Solutions:* Apache HBase/Hadoop is based on Google's BigTable distributed storage system, which runs on top of Hadoop as a distributed and scalable big data store. This means that HBase can leverage the distributed processing paradigm of the Hadoop Distributed File System (HDFS) and benefit from Hadoop's MapReduce programming model. It combines the scalability of Hadoop with real-time data access as a key/value store and deep analytic capabilities of Map Reduce [4].HBase allows to query for individual records as well as derive aggregate analytic reports across a massive amount of data. It can host large tables with billions of rows, millions of columns and run across a cluster of commodity hardware. HBase is composed of three types of servers in a master slave type of architecture. Region servers are responsible to serve data for reads and writes. When accessing data, clients communicate with HBase Region Servers directly. Region assignment, DDL (create, delete tables) operations are handled by the HBase Master process.* Apache Cassandra is a distributed database used for the administration and management of large amounts of structured data across multiple servers, while providing highly available service and no single point of failure. It provides features such as continuous availability, linear scale performance, data distribution across multiple data centers and cloud availability zones. Cassandra inherits its data architecture from Google's BigTable and it borrows its distribution mechanisms from Amazon's Dynamo.The nodes in a Cassandra cluster are completely symmetrical, all having identical responsibilities. Cassandra also employs consistent hashing to partition and replicate data. it has the capability of handling large amounts of data and thousands of concurrent users or operations per second across multiple data centers.Cassandra has a hierarchy of caching mechanisms and carefully orchestrated disk I/O which ensures speed and data safety. Write operations are sent first to a persistent commit log (ensuring a durable write), then to a write-back cache called a memTable; when the memTable fills, it is flushed to a sorted string table - SSTable - on disk. A Cassandra cluster is organized as a ring, and it uses a partitioning strategy to distribute data evenly.* Redis represents an in-memory data structure store used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperlogs and geospatial indexes with radius queries. Redis stores all data in RAM, allowing lightning fast reads and writes. It runs extremely efficiently in memory and handles high-velocity data, needing simple standard servers to deliver millions of operations per second with submillisecond latency. …

RESeED: A secure regular‐expression search tool for storage clouds

SummaryLack of trust has become one of the main concerns of users who tend to utilize one or multiple Cloud providers. Trustworthy Cloud‐based computing and data storage require secure and efficient solutions which allow clients to remotely store and process their data in the Cloud. User‐side encryption is an established method to secure the user data on the Cloud. However, using encryption, we lose processing capabilities, such as searching, over the Cloud data. In this paper, we present RESeED, a tool that provides user‐transparent and Cloud‐agnostic regular‐expression search functionality over encrypted data across multiple Clouds. Upon a client's intent to upload a new document to the Cloud, RESeED analyzes the document's content and updates its data structures accordingly. Then, it encrypts and transfers the document to the Cloud. RESeED provides the regular‐expression search functionality over encrypted data by translating the search queries on‐the‐fly to finite automata and analyzing concise and secure representations of the data before asking the Cloud to download the encrypted documents. RESeED's parallel architecture enables efficient search over large‐scale (and potentially big data scale) data‐sets. We evaluate the performance of RESeED experimentally and demonstrate its scalability and correctness using real‐world data‐sets from arXiv.org and Internet Engineering Task Force (IETF). Our results show that RESeED produces accurate query responses with a reasonable (≃6%) storage overhead. The results also demonstrate that for many search queries, RESeED performs faster in compare with the grep utility that functions on unencrypted data. Copyright © 2017 John Wiley & Sons, Ltd.

Unsettling the ‘friendly’ gaze of dataveillance: the dissident potential of mediatised aesthetics in Blast Theory’s Karen

ABSTRACTWhat are the artistic languages and forms that can be used to make sense of the larger-than-human scale of big data and engage with its ideological machinery? How can theatre and performance in a mediatised culture disclose the performativity of dataveillance and open spaces for thinking differently and critically about it? Blast Theory’s interactive, virtual theatre piece Karen (2015), which is formed through a smartphone app and is communicated individually to its participants on their phones, addresses such questions. Karen is designed to mine data from the participants, which is then used to profile each of them through a personalised data report. Blast Theory’s piece, on the one hand, offers a familiar, interactive and participatory experience, generating a sense of agency and control. On the other hand, it reminds the participants that they are not in control of their own data by making the familiar experience strange and subverting the performativity of surveillance. Drawing on and combining the notions of mediatisation and info-aesthetics, this article argues that through its ‘mediatised aesthetics’ Karen provokes critical recognition, challenging our habitual understandings of data surveillance, and illustrates a paradigm-in-progress to explore the new aesthetics of the mediatised age.

Development of a Total Environment Data Science Approach in a Big Data Scale

We use the Big Data paradigm, as a driving mechanism of an integrated research framework. As a case study, we consider analysing various ecological systems and their connectivity in the framework. An unknown coexistence among different species and lack of knowledge on their sustainability motivate us for undertaking the current research. For describing the recycling systems in nature, an articulated design science research (DSR) framework is necessary for which we have constructed data models for composite lithosphere-atmosphere-biosphere-hydrosphere ecosystem (LABHE). The unstructured big-size environmental data sources and their anomalies existing in nature are taken advantage of, to describe various constructs, compute models and validate them by DSR guidelines. For this purpose, the domain ontology artefacts are drawn and integrated into a warehouse approach to compute an environmental metadata and interpret it in different knowledge domains. The data models and the proposed integrated framework facilitate the environment explorers for planning and management of environmental resources worldwide. The Big Data associated LABHE bring out new knowledge and its interpretation in a variety of environmental data map and plot views. The constructs, models and methodologies used in the current domain application are research deliverables for the total environment researchers and explorers.

Data Visualization as Documentary Form: The Murmur of Digital Magnitude

Data Visualization as Documentary Form:The Murmur of Digital Magnitude Tess Takahashi (bio) In the last decade, data visualization has emerged as a form of documentary. While data visualization is not typically thought of as being in the same category as documentary film, these visually pleasing and often amusing depictions of data culled from the natural and human world pop up repeatedly on our web feeds, often over geographic maps of the world that are detached from their larger context. In other words, data visualization is a rhetorical form that visualizes a relationship between the material world and its particular form of representation. At a time when both documentary forms in the art world and popular feature-length documentaries have flourished, data visualizations have become increasingly popular and powerful forms of documentary. It goes without saying that data visualizations are primarily visual, not aural, representations. Rather than documentary cinema's authoritative voice-over or the voicings of authentic witness, data visualization is typically silent. Yet I argue that data visualization not only has a metaphorical "voice," like the voice of documentary film, but also claims to represent the literal material "voices" of others, even if those voices are seen as inarticulate, unrecognizable, or impossible—the voice of Earth, for example. This essay [End Page 376] asks how we might figure the documentary voice of data visualization as a metaphorical form that "speaks" to the eye. Further, it asks what thinking sonically about data visualization might reveal about voice and documentary. At stake here is what becomes of material embodied voices as they are absorbed into the authoritative voice of what is now our most ubiquitous documentary form: the data visualization. Today, data visualizations often draw on a wealth of "Big Data," which Steve Lohr has described as "the rising flood of digital data from many sources, including the Web, biological and industrial sensors, video, e-mail and social network communications."1 In the face of the incomprehensible scale of Big Data, data visualizations present an image that squeezes data from the big world into the instant of a glance—decades of rising sea levels, one hundred years of increasing temperatures, the extinction of species after species—by drawing on a combination of numerical measurement and photographic evidence. The most common contemporary forms of data visualization are static maps or looping, bite-size gifs, but data visualization has a longer history of attempts to make enormous amounts of data graspable in a glance. This tradition includes the work of Scottish engineer William Playfair, who in the late 1700s created the bar chart, the pie chart, and the line graph—forms that, in the words of nineteenth-century civil engineer Charles Minard, can be used to "speak to the eyes."2 At that time, the supposedly immediate form of communication offered by charts and graphs constituted "visual breakthroughs, innovations that allowed people to see patterns in data that they would otherwise have missed if they just stared at long tables of numbers."3 In more recent history, from the 1980s through the early 2000s, MIT professor Edward Tufte's three influential books on data visualization have chronicled the numerous ways that we have envisioned information over past centuries, ranging from hand-drawn hatch marks over sailors' maps to suggest the turbulence of the wind to clunky, artifact-ridden digital clouds generated by 1990s computer software.4 Today in the age of Big Data, the sophistication of data visualization tools has grown by orders of magnitude, like the sheer volume of data in the world. If the amount of data generated from the beginning of recorded time to 2003 rings in at a measly five billion gigabytes, we now generate that amount every two days.5 How might what William Davies describes as the now "default" collection of Big Data change what we think of as a "voice," and what counts as "speech" for data visualization as a documentary form?6 While data visualization allows the eye to take in large amounts of data in the space of a glance, the embodied human voice cannot [End Page 377] be taken into the ear "at a glance." Steven Connor has described the space of...

Firemap: A Dynamic Data-Driven Predictive Wildfire Modeling and Visualization Environment

Wildfires are destructive fires over the wildland that can wipe out large areas of vegetation and infrastructure. Such fires are hard to control and manage as they can change directions almost instantly, driven by changing environmental conditions. Effective response to such events requires the ability to monitor and predict the behavior of the fire as fast as they change. The WIFIRE project builds an end-to-end cyberinfrastructure for real-time and data-driven simulation, prediction, and visualization of wildfire behavior. One goal of WIFIRE is to provide the tools to predict a more accurate rate of a spreading wildfire. To this end, WIFIRE has developed interfaces for ingesting and visualizing high-density sensor networks to improve fire and weather predictions, and has created a data model for wildfire resources including sensed and archived data, sensors, satellites, cameras, modeling tools, workflows, and social information including Twitter feeds for wildfire research and response. This paper presents WIFIRE’s Firemap web platform to make these geospatial data and products accessible. Through a web browser, Firemap enables geospatial information visualization and a unified access to geospatial workflows using Kepler. Using GIS capabilities combined with scalable big data integration and processing, Firemap enables simple execution of the model with options for running ensembles by taking the information uncertainty into account. The results are easily viewable, sharable, repeatable, and can be animated as a time series.

Multi-source uncertain entity resolution: Transforming holocaust victim reports into people

In this work we present a multi-source uncertain entity resolution model and show its implementation in a use case of Yad Vashem, the central repository of Holocaust-era information. The Yad Vashem dataset is unique with respect to classic entity resolution, by virtue of being both massively multi-source and by requiring multi-level entity resolution. With today's abundance of information sources, this project motivates the use of multi-source resolution on a big-data scale. We instantiate the proposed model using the MFIBlocks entity resolution algorithm and a machine learning approach, based upon decision trees to transform soft clusters into ranked clustering of records, representing possible entities. An extensive empirical evaluation demonstrates the unique properties of this dataset that make it a good candidate for multi-source entity resolution. We conclude with proposing avenues for future research in this realm.

Simulation concept of NICA-MPD-SPD Tier0-Tier1 computing facilities

The simulation concept for grid-cloud services of contemporary HENP experiments of the Big Data scale was formulated in practicing the simulation system developed in LIT JINR Dubna. This system is intended to improve the efficiency of the design and development of a wide class of grid-cloud structures by using the work quality indicators of some real system to design and predict its evolution. For these purposes the simulation program is combined with a real monitoring system of the grid-cloud service through a special database (DB). The DB accomplishes acquisition and analysis of monitoring data to carry out dynamical corrections of the simulation. Such an approach allows us to construct a general model pattern which should not depend on a specific simulated object, while the parameters describing this object can be used as input to run the pattern. The simulation of some processes of the NICA-MPD-SPD Tier0-Tier1 distributed computing is considered as an example of our approach applications.

Toward large-scale crop production forecasts for global food security

Predicting crop production plays a critical role in food price forecasting and mitigating potential food shortages. Crop models may require parameters from, for example, weather, crop genotype, farm management, and soil. Sources for these data are often found in very different places. Researchers spend a significant amount of time to collect and curate them. In addition, in order to scale yield forecasts from the single-farm level up to the continental scale, crop models have to be coupled with a geospatial big data platform to provide the required data inputs. In a proof-of-concept case study, we investigate the coupling of a scalable geospatial big data platform, Physical Analytics Integrated Repository and Services (PAIRS), to the Decision Support System for Agrotechnology Transfer (DSSAT) crop model. We envision running this system on a global scale. For geospatial analytics, PAIRS provides curation of heterogeneous data sources to simulate crop models using hundreds of terabytes of data.

Size Matters to Lesbians Too: Queer Feminist Interventions into the Scale of Big Data

How can we recognize those whose lives and data become attached to the far-from-groundbreaking framework of “small data”? Specifically, how can marginalized people who do not have the resources to produce, self-categorize, analyze, or store “big data” claim their place in the big data debates? I examine the place of lesbians and queer women in the big data debates through the Lesbian Herstory Archive’s not “big” enough lgbtq organizing history dataset — perhaps the largest dataset known to exist on lgbtq activist history — as one such alternative. A contribution to critical data studies, I take a queer feminist approach to the scale of big data by reading for the imbricated scales and situated knowledge of data.

Population-Based MCMC on Multi-Core CPUs, GPUs and FPGAs

Markov Chain Monte Carlo (MCMC) is a method to draw samples from a given probability distribution. Its frequent use for solving probabilistic inference problems, where big-scale data are repeatedly processed, means that MCMC runtimes can be unacceptably large. This paper focuses on population-based MCMC, a popular family of computationally intensive MCMC samplers; we propose novel, highly optimized accelerators in three parallel hardware platforms (multi-core CPUs, GPUs and FPGAs), in order to address the performance limitations of sequential software implementations. For each platform, we jointly exploit the nature of the underlying hardware and the special characteristics of population-based MCMC. We focus particularly on the use of custom arithmetic precision, introducing two novel methods which employ custom precision in the largest part of the algorithm in order to reduce runtime, without causing sampling errors. We apply these methods to all platforms. The FPGA accelerators are up to 114x faster than multi-core CPUs and up to 53x faster than GPUs when doing inference on mixture models.

An integrated text mining framework for metabolic interaction network reconstruction.

Text mining (TM) in the field of biology is fast becoming a routine analysis for the extraction and curation of biological entities (e.g., genes, proteins, simple chemicals) as well as their relationships. Due to the wide applicability of TM in situations involving complex relationships, it is valuable to apply TM to the extraction of metabolic interactions (i.e., enzyme and metabolite interactions) through metabolic events. Here we present an integrated TM framework containing two modules for the extraction of metabolic events (Metabolic Event Extraction module—MEE) and for the construction of a metabolic interaction network (Metabolic Interaction Network Reconstruction module—MINR). The proposed integrated TM framework performed well based on standard measures of recall, precision and F-score. Evaluation of the MEE module using the constructed Metabolic Entities (ME) corpus yielded F-scores of 59.15% and 48.59% for the detection of metabolic events for production and consumption, respectively. As for the testing of the entity tagger for Gene and Protein (GP) and metabolite with the test corpus, the obtained F-score was greater than 80% for the Superpathway of leucine, valine, and isoleucine biosynthesis. Mapping of enzyme and metabolite interactions through network reconstruction showed a fair performance for the MINR module on the test corpus with F-score >70%. Finally, an application of our integrated TM framework on a big-scale data (i.e., EcoCyc extraction data) for reconstructing a metabolic interaction network showed reasonable precisions at 69.93%, 70.63% and 46.71% for enzyme, metabolite and enzyme–metabolite interaction, respectively. This study presents the first open-source integrated TM framework for reconstructing a metabolic interaction network. This framework can be a powerful tool that helps biologists to extract metabolic events for further reconstruction of a metabolic interaction network. The ME corpus, test corpus, source code, and virtual machine image with pre-configured software are available at www.sbi.kmutt.ac.th/ preecha/metrecon.

Biomedical Big Data Training Collaborative (BBDTC): An Effort to Bridge the Talent Gap in Biomedical Science and Research

The BBDTC (https://biobigdata.ucsd.edu) is a community-oriented platform to encourage high-quality knowledge dissemination with the aim of growing a well-informed biomedical big data community through collaborative efforts on training and education. The BBDTC collaborative is an e-learning platform that supports the biomedical community to access, develop and deploy open training materials. The BBDTC supports Big Data skill training for biomedical scientists at all levels, and from varied backgrounds. The natural hierarchy of courses allows them to be broken into and handled as modules. Modules can be reused in the context of multiple courses and reshuffled, producing a new and different, dynamic course called a playlist. Users may create playlists to suit their learning requirements and share it with individual users or the wider public. BBDTC leverages the maturity and design of the HUBzero content-management platform for delivering educational content. To facilitate the migration of existing content, the BBDTC supports importing and exporting course material from the edX platform. Migration tools will be extended in the future to support other platforms. Hands-on training software packages, i.e., toolboxes, are supported through Amazon EC2 and Virtualbox virtualization technologies, and they are available as: (i) downloadable lightweight Virtualbox Images providing a standardized software tool environment with software packages and test data on their personal machines, and (ii) remotely accessible Amazon EC2 Virtual Machines for accessing biomedical big data tools and scalable big data experiments. At the moment, the BBDTC site contains three open Biomedical big data training courses with lecture contents, videos and hands-on training utilizing VM toolboxes, covering diverse topics. The courses have enhanced the hands-on learning environment by providing structured content that users can use at their own pace. A four course biomedical big data series is planned for development in 2016.