Today's Big Research Articles

Security Process in Hadoop Using Diverse Approach

Today, data is one of the most important assets for nearly every sector of industry. Advances in information technology and widespread growth in various fields such as business, engineering, medicine, and scientific research have resulted in an explosion of data. This data explosion presents new challenges that traditional methods cannot handle. One of the biggest concerns of our time concerns the security and protection of sensitive information. In today's Big Data era, where organizations collect, analyse, and make decisions based on analysis of massive datasets from various sources, security in this process becomes more and more important. At the same time, more and more organizations need to enforce access controls and privacy restrictions on these records to meet regulatory requirements such as HIPAA and other privacy laws. Network security breaches by internal and external attackers are on the rise, often taking months to discover, and those affected pay the price. Organizations that fail to properly control access to records face lawsuits, negative publicity, and fines.

Open Data in Catalysis: From Today's Big Picture to the Future of Small Data

AbstractOpen science and data are yet to make a real breakthrough and research policies will have a critical role in it. The history and general context around open data is hence firstly addressed, including how researchers perceive the existing incentives, leading to recommendations on how to foster data sharing. Subsequently, the focus is on catalysis, with a particular emphasis on benchmarking the data sharing practices against other fields and surveying the type of data currently being shared. The current infrastructure, including data repositories, and standards formats is maped. The striking differences among different disciplines are discussed, serving as a basis to propose specific actions to promote data sharing in catalysis. Short‐term initiatives are needed to boost the amount of openly available data, particularly in heterogeneous catalysis, but a high degree of standardization in data formats will be needed to ensure optimal and automated data mining in the long run. Because of its unique, central role in understanding the catalytic action, kinetic catalytic data is of particular interest. As formats and mining tools are dependant on the type of data, kinetic catalytic data is firstly characterized. Guidelines for a standardized sharing format are proposed, taking into account the small, well‐structured nature of this type of data. To maximize the extraction of information, the low volume of kinetic catalytic data will be compensated by incorporating fundamental knowledge into statistics‐based tools. Whencoupled with knowledge generation tools, i. e. kinetic models, new insights at the active site and mechanism levels will be reached in an ever more automated and powerful way.

BAD to the bone: Big Active Data at its core

Virtually all of today's Big Data systems are passive in nature, responding to queries posted by their users. Instead, we are working to shift Big Data platforms from passive to active. In our view, a Big Active Data (BAD) system should continuously and reliably capture Big Data while enabling timely and automatic delivery of relevant information to a large pool of interested users, as well as supporting retrospective analyses of historical information. While various scalable streaming query engines have been created, their active behavior is limited to a (relatively) small window of the incoming data. To this end we have created a BAD platform that combines ideas and capabilities from both Big Data and Active Data (e.g., Publish/Subscribe, Streaming Engines). It supports complex subscriptions that consider not only newly arrived items but also their relationships to past, stored data. Further, it can provide actionable notifications by enriching the subscription results with other useful data. Our platform extends an existing open-source Big Data Management System, Apache AsterixDB, with an active toolkit. The toolkit contains features to rapidly ingest semistructured data, share execution pipelines among users, manage scaled user data subscriptions, and actively monitor the state of the data to produce individualized information for each user. This paper describes the features and design of our current BAD data platform and demonstrates its ability to scale without sacrificing query capabilities or result individualization.

Today's Big Three (Problems)

Today's Big Three (Problems)

Robust and efficient memory management in Apache AsterixDB

SummaryTraditional relational database systems handle data by dividing their memory into sections such as a buffer cache and working memory, assigning a memory budget to each section to efficiently manage a limited amount of overall memory. They also assign memory budgets to memory‐intensive operators such as sorts and joins and control the allocation of memory to these operators; each memory‐intensive operator attempts to maximize its memory usage to reduce disk I/O cost. Implementing such memory‐intensive operators requires a careful design and application of appropriate algorithms that properly utilize memory. Today's Big Data management systems need the ability to handle large amounts of data similarly, as it is unrealistic to assume that truly big data will fit into memory. In this article, we share our memory management experiences in Apache AsterixDB, an open‐source Big Data management software platform that scales out horizontally on shared‐nothing commodity computing clusters. We describe the implementation of AsterixDB's memory‐intensive operators and their designs related to memory management. We also discuss memory management at the global (cluster) level. We conducted an experimental study using several synthetic and real datasets to explore the impact of this work. We believe that future Big Data management system builders can benefit from these experiences.

Big Data from A to Z

The rapid development of technology over the past 20 years has led to explosive data growth in various industries, including defense industries, healthcare. The analysis of generated Big Data has recently been addressed by many researchers, because today's Big Data analysis are one of the most important and most profitable areas of development in Data Science and companies that are able to extract valuable knowledge among the massive amount of data at logical time can earn significant advantages . Accordingly, in this survey, we investigate definition of the Big Data and the data sources. Also look at advantages, challenges, applications, analysis and platforms used in the Big Data.

The Second Green Revolution Will Bring Agri-Tech Breakthroughs to Growers

The Second Green Revolution Will Bring Agri-Tech Breakthroughs to Growers

A BAD demonstration

Nearly all of today's Big Data systems are passive in nature. We demonstrate our Big Active Data ("BAD") system, a scalable system that continuously and reliably captures Big Data and facilitates the timely and automatic delivery of new information to a large population of interested users as well as supporting analyses of historical information. We built our BAD project by extending an existing scalable, open-source BDMS (AsterixDB [1]) in this active direction. In this demonstration, we allow our audience to participate in an emergency notification application built on top of our BAD platform, and highlight its capabilities.

Entity reconciliation in big data sources: A systematic mapping study

The entity reconciliation (ER) problem aroused much interest as a research topic in today's Big Data era, full of big and open heterogeneous data sources. This problem poses when relevant information on a topic needs to be obtained using methods based on: (i) identifying records that represent the same real world entity, and (ii) identifying those records that are similar but do not correspond to the same real-world entity. ER is an operational intelligence process, whereby organizations can unify different and heterogeneous data sources in order to relate possible matches of non-obvious entities. Besides, the complexity that the heterogeneity of data sources involves, the large number of records and differences among languages, for instance, must be added. This paper describes a Systematic Mapping Study (SMS) of journal articles, conferences and workshops published from 2010 to 2017 to solve the problem described before, first trying to understand the state-of-the-art, and then identifying any gaps in current research. Eleven digital libraries were analyzed following a systematic, semiautomatic and rigorous process that has resulted in 61 primary studies. They represent a great variety of intelligent proposals that aim to solve ER. The conclusion obtained is that most of the research is based on the operational phase as opposed to the design phase, and most studies have been tested on real-world data sources, where a lot of them are heterogeneous, but just a few apply to industry. There is a clear trend in research techniques based on clustering/blocking and graphs, although the level of automation of the proposals is hardly ever mentioned in the research work.

Real-time stream processing for Big Data

Abstract With the rise of the web 2.0 and the Internet of things, it has become feasible to track all kinds of information over time, in particular fine-grained user activities and sensor data on their environment and even their biometrics. However, while efficiency remains mandatory for any application trying to cope with huge amounts of data, only part of the potential of today's Big Data repositories can be exploited using traditional batch-oriented approaches as the value of data often decays quickly and high latency becomes unacceptable in some applications. In the last couple of years, several distributed data processing systems have emerged that deviate from the batch-oriented approach and tackle data items as they arrive, thus acknowledging the growing importance of timeliness and velocity in Big Data analytics. In this article, we give an overview over the state of the art of stream processors for low-latency Big Data analytics and conduct a qualitative comparison of the most popular contenders, namely Storm and its abstraction layer Trident, Samza and Spark Streaming. We describe their respective underlying rationales, the guarantees they provide and discuss the trade-offs that come with selecting one of them for a particular task.

Breaking BAD: A Data Serving Vision for Big Active Data.

Virtually all of today's Big Data systems are passive in nature. Here we describe a project to shift Big Data platforms from passive to active. We detail a vision for a scalable system that can continuously and reliably capture Big Data to enable timely and automatic delivery of new information to a large pool of interested users as well as supporting analyses of historical information. We are currently building a Big Active Data (BAD) system by extending an existing scalable open-source BDMS (AsterixDB) in this active direction. This first paper zooms in on the Data Serving piece of the BAD puzzle, including its key concepts and user model.

Microtrends: The Small Forces behind Today's Big Changes

Microtrends: The Small Forces behind Today's Big Changes