Big Data Term Research Articles

Big Data and Hadoop Architecture: A Review Paper

In the world of information, the term Big Data comes with new opportunity and challenges to deal with massive amount of data. Big data terms as large, diverse sets of information that grow at exponential rates. 90% of data is come within past 2-3 years. So, it is big responsibility to take care of this data and retrieve some useful information from it. To find useful information from huge amount of data, we need to analyze the data. In this review paper presents overview of big data, use of Hadoop architecture in big data, scope for the future research. This paper discusses about the big data types after that we discuss Hadoop and its mechanism. Keyword: Big Data, Hadoop, MapReduce, HDFS, Yarn

What is “big data” and how should we use it? The role of large datasets, secondary data, and associated analysis techniques in outdoor recreation research

With researchers increasingly interested in big data research, this conceptual paper describes how large datasets, secondary data, and associated analysis techniques can be used to understand outdoor recreation. Some types of large, secondary datasets that have been increasingly used in outdoor recreation research include social media, mobile device data, and trip reports or online reviews. First, we give a brief overview of big data terms and outline the steps involved in conducting big data research. In doing so, we describe data sources and analysis techniques relevant for outdoor recreation, and review how they have been applied in previous published works. We then describe opportunities, limitations, and considerations of using big data. Finally, we outline several questions researchers may consider when designing, conducting, reporting, and reviewing outdoor recreation research using big data. Overall, big data approaches can expand our understanding of outdoor recreation and, by addressing key questions, may help researchers harness the strengths of big data while ensuring quality and integrity.

Discussion of “Probability vs. Nonprobability Sampling: From the Birth of Survey Sampling to the Present Day” by Graham Kalton

Let me first thank Dr. Kalton for his amazing historical review of the development of survey sampling from its origin, contrasting purposive sampling, until now, where some elements of purposive sampling in terms of web or big data seem to supersede the well-elaborated theory of survey statistics. Shall the message be that we do not need any sampling courses at universities anymore, that official statistics should turn to modelling using data with unknown data generating processes, or actually even be substituted by (commercial) data krakens? Hardly so! Graham Kalton emphasises a modern thinking about the use of these new data sources which may also have some advantages and he urges future research on data integration methods using (very) different kinds of data while strongly taking quality aspects into account.

SM2-Based Offline/Online Efficient Data Integrity Verification Scheme for Multiple Application Scenarios.

With the rapid development of cloud storage and cloud computing technology, users tend to store data in the cloud for more convenient services. In order to ensure the integrity of cloud data, scholars have proposed cloud data integrity verification schemes to protect users' data security. The storage environment of the Internet of Things, in terms of big data and medical big data, demonstrates a stronger demand for data integrity verification schemes, but at the same time, the comprehensive function of data integrity verification schemes is required to be higher. Existing data integrity verification schemes are mostly applied in the cloud storage environment but cannot successfully be applied to the environment of the Internet of Things in the context of big data storage and medical big data storage. To solve this problem when combined with the characteristics and requirements of Internet of Things data storage and medical data storage, we designed an SM2-based offline/online efficient data integrity verification scheme. The resulting scheme uses the SM4 block cryptography algorithm to protect the privacy of the data content and uses a dynamic hash table to realize the dynamic updating of data. Based on the SM2 signature algorithm, the scheme can also realize offline tag generation and batch audits, reducing the computational burden of users. In security proof and efficiency analysis, the scheme has proven to be safe and efficient and can be used in a variety of application scenarios.

A New Centralized Detection-Based Process for Evaluating Anomalies and Analyzing the First Causes Using Machine Learning and Web Semantic

In the last decades, many works have been done to enhance data performances in the computer field. Data performance consists to describe all improvements which can be added to data traffic. More precisely, we are talking about techniques allowing improving the evaluation of big data using machine learning. Data evaluation is composed of several variables such as security, quality of service, data synchronization, scalability, and data structuring. In this work, we complete our proceedings done to supervise the continuity of technological evolution in terms of big data and safety. In other words, we aim to add brick to our previous processes to take into consideration the enhancement of the analysis of the causes generating frauds and intrusions preventing data traffic. To achieve this end, we increase current machine learning techniques with prior knowledge based on data thresholds set by experts in the first place. We also aim to integrate knowledge facilitating the interpretation of the causes causing all kinds of anomalies in the second place. Finally, our process will be endowed with the requirements to improve the rate of detection of anomalies and reduce human involvement operation.

Big Data, Sentiment Analysis, and Examples during the COVID-19 Pandemic

Applied research in Big Data has gained popularity and is already transforming corporations, public sector, health care and subsequently everyday life. Big Data are being analysed for a variety of reasons, e.g., predict Brexit negotiating outcomes, optimise operations in agriculture, map and analyse human mobility trends under non-pharmaceutical interventions during the recent pandemic. The period of the COVID-19 pandemic has been characterised also by an ‘infodemic’, meaning an overabundance of both good and bad information. This information needs to be managed effectively as it can yield valuable insights when analysed. In this paper the terms of Big Data, Geospatial Big Data and Sentiment Analysis are presented along with selected cases, from the international literature, of the use of Big Data and analytics during the COVID-19 pandemic.

Performance Analysis of Bloom Filter for Big Data Analytics.

The rapid rise of data value, such as social media and mobile applications, results in large volumes of data, which is what the term "big data" refers to. The increased rate of data growth makes handling big data very challenging. Despite a Bloom filter (BF) technique having previously been proposed as a space-and-time efficient probabilistic method, this proposal has not yet been evaluated in terms of big data. This study, thus, evaluates the BF technique by conducting an experimental study with a large amount of data. The results revealed that BF overcomes the efficiency not present in the space-and-time of indexing and examining big data. Moreover, to address the increase of false-positive rate in using BF with big data, a novel false-positive rate reduction approach is proposed in this paper. The initial experimental results of evaluating this method are very promising. The novel approach helped to reduce the false-positive rate by more than 70%.

Open-source intelligence for conservation biology.

Open-source intelligence (OSINT) evolved in spy agencies but now is rapidly changing many fields of study, from anthropology to zoology. Despite the fact that OSINT occasionally is used in conservation biology, there is little recognition that some tools and frameworks used by conservation professionals are drawn from this well-established field. In conservation biology, OSINT is sometimes used to evaluate wildlife crime, human-wildlife and human-environment interactions, animal behavior, and questions of distribution and abundance. Recognizing the conceptual foundations of the field would allow expansion of conservation biology, not only in the areas noted above, but also, for example, in study of habitat use, habitat change, and animal behavior. This recognition would also provide frameworks for conceptual advancement, especially in terms of data and privacy management. Failure to recognize the underpinnings of OSINT tools in conservation biology harms the field because it limits how research is framed, thought about, and implemented. Likewise, taking an OSINT perspective to conservation problems, rather than simply thinking in terms of big data, can enrich the field, expand science, and increase knowledge and understanding of biology and biodiversity.

Application Research of Advanced Intelligent Big Data Analysis Based on Intelligent Sensor Network in the Design of Personalized Education Management System and the Construction of Innovation System

With the continuous innovation and development of information technology and communication technology, human beings have entered a huge data center. In terms of big data, in-depth discussion on the in-depth application of big data is the current situation and development direction. Based on the intelligent sensor network, this paper conducts theoretical and empirical analysis combined with the application of individualized teaching in teaching and provides some reference opinions for the future development of individual pedagogy. On the basis of the research, the meaning of big data is explained from multiple perspectives. In the empirical research part, based on big data, based on real big data, and using relevant technologies in data mining, the relevance of each training program is analyzed. Mining was carried out using the FP-Growth method to analyze the correlation of each training scheme and optimize the FP-Growth method accordingly. The results of this experiment show that the use of big data can not only help teachers understand their students’ daily life, study habits, and consumption records but also make teachers’ work more dynamic, early warning, and subsidized. The research results are of great significance for guiding colleges and universities to formulate educational programs that are beneficial to students’ development, keep pace with the times, and cultivate modern educational technicians with modern significance.

Classification of medical specialty for text medical report based on natural language processing and deep learning

Natural language processing (NLP) is a part of artificial intelligence algorithms that focus on designing and building applications and systems in a way that allows interaction between computers and natural languages ​​developed for human use. NLP has been used in several areas within artificial intelligence and data processing applications such as social media applications and medical applications and translation applications. The medical field is one of the richest in terms of big data, which has not been well invested so far. one of these unused data was the text medical reports. Natural Language Processing analysis is an effective way of examining medical reports and social media data to improve treatments and patient services. by training a computer to make a decision instead of a human, as soon as possible from the decisions of specialist doctors or medical staff working in health institutions to diagnose diseases or Prescribe treatment to the patient. The data of each patient is stored in the form of medical reports mostly as a text document file and kept within the hospital data. Hospitals suffer from a scarcity of medical and nursing staff, in addition to the high cost of hiring medical staff.

Rifiuti informatici, inquinamento digitale. Il lato insostenibile della 4a rivoluzione industriale

This paper is devoted to the electronic waste and the digital pollution topic. Firstly, concerning the e-waste, we are going to see its origin, the directions of its disposal, and if there is any possibility of recovery. Secondly, we are going to see the digital pollution. The data exchanged in internet leave a footprint not only in terms of Big Data but also in terms of the CO2 emissions. Millions of physical servers in data centers around the world, are connected with miles of undersea cables, and with switches and routers. All of them require a lot of energy to run. Finally, we will examine what measures governments can promote to manage the digital and ecological transitions. The challenge is to implement the digitalization process in the sense of sustainability according to the EU economic and social policies.

Application and Effectiveness of Big Data and Artificial Intelligence in the Construction of Nursing Sensitivity Quality Indicators.

In order to explore the quality management efficiency of applying big data and artificial intelligence in nursing quality index, a method of building a nursing management platform integrating nursing indicators and nursing events is proposed. Based on the investigation of the application demand of nursing information system, the method achieves timely data sharing and transmission through WLAN technology and realizes nursing management monitoring, nursing quality index enquiry, and automatic statistical analysis under the vertical management mode of nursing. The results showed that 77 people (73%) thought the time decreased, 19 people (18%) thought the time was the same, and 9 people (7%) thought the time increased. In terms of intelligent application and big data of nursing information management system, there is a significant difference in nursing management efficiency before and after using nursing management information system (P < 0.001). The nursing management control platform is designed and applied, and the nursing quality control method and actual management process are improved, which is very good for strengthening nursing quality management. The overall optimization of the quality control process is realized, which helps to mobilize the initiative and enthusiasm of nursing staff and continuously improve the effectiveness of nursing management and nursing efficiency.

Strategic issues of big data analytics applications for managing health-care sector: a systematic literature review and future research agenda

PurposeThe proposed article is aimed at exploring the opportunities, challenges and possible outcomes of incorporating big data analytics (BDA) into health-care sector. The purpose of this study is to find the research gaps in the literature and to investigate the scope of incorporating new strategies in the health-care sector for increasing the efficiency of the system.Design/methodology/approachFora state-of-the-art literature review, a systematic literature review has been carried out to find out research gaps in the field of healthcare using big data (BD) applications. A detailed research methodology including material collection, descriptive analysis and categorization is utilized to carry out the literature review.FindingsBD analysis is rapidly being adopted in health-care sector for utilizing precious information available in terms of BD. However, it puts forth certain challenges that need to be focused upon. The article identifies and explains the challenges thoroughly.Research limitations/implicationsThe proposed study will provide useful guidance to the health-care sector professionals for managing health-care system. It will help academicians and physicians for evaluating, improving and benchmarking the health-care strategies through BDA in the health-care sector. One of the limitations of the study is that it is based on literature review and more in-depth studies may be carried out for the generalization of results.Originality/valueThere are certain effective tools available in the market today that are currently being used by both small and large businesses and corporations. One of them is BD, which may be very useful for health-care sector. A comprehensive literature review is carried out for research papers published between 1974 and 2021.

Industry 4.0 Adaption in Indian Banking Sector—A Review and Agenda for Future Research

Digitalization and intelligization is the need of the hour in today’s world. The manufacturing industry is, in fact, moving towards the fourth-generation industry, which we termed as Industry 4.0 or the Fourth Industrial revolution, which is defined as a new level of organization and control over the entire value chain of the life cycle of products; it is geared towards increasingly individualized customer requirements. Industry 4.0 is all about talking in terms of big data, technology, cyber security, the Internet of Things (IoT) and so on. This study is done to understand the new emerging technology in data exchange and automation, popularly known as Industry 4.0, in terms of banking sector with context to the Indian banking sector. The study focuses on studying banks in a digitalized word and what are the challenges that banks face. How banks cope up with digitalization, keeping customers at priority. This study centred on incorporating articles published in recent years to establish knowledge on the topic and to further identify areas for future research.

Construction and Application of Smart Education Teaching Platform

With the continuous development of science and technology, my country has gradually entered the era of big data. In this case, various Internet technologies and teaching software continue to appear and are widely used in classrooms. Therefore, this article discusses how to make better use of intelligent learning platforms in terms of big data to improve the quality of education and serve platform users. This system divides teaching work into basic modules and provides a network application platform for smart classrooms. Through the connection with the education cloud platform, it can identify the sharing of high-quality educational resources, expand its implementation and promotion, and use intelligent learning. The platform is the test group, and traditional education is the control group for all research fields and research. The research results show that the intelligent learning platform is a new type of service platform that has a certain impact on the traditional education model. However, from another perspective, the technology has improved the traditional education model in an innovative way. This model can be used to realize intelligent learning through the use of Internet technology, and can solve various problems in the field of education, increase the allocation of reasonable educational resources and significantly improve the quality of education.

A Hybrid Authentication Policy for Digital Image Processing in Big Data Platform

Big data term includes all types of structured and unstructured data in its database. The extraction of data from big data can be done through Map-Reduce, Hadoop and many more. Now a days, Digital image processing is needing its data security due to its use in widespread firms e.g. (institutions, corporate companies, marketing, streaming etc.) in the form of fingerprint, palmprint, face- recognition etc. The matching and compression process of images is not enough today to secure it. This paper proposes a hybrid authentication framework used in Big data for various types of image data. This framework also used a cryptographic approach with Local Binary Patterns Technique (LBP) image data to ensure better security.

A comparative study between artificial bee colony (ABC) algorithm and its variants on big data optimization

The big data term and its formal definition have changed the properties of some of the computational problems. One of the problems for which the fundamental properties change with the existence of the big data is the optimization problems. Artificial bee colony (ABC) algorithm inspired by the intelligent source search, consumption and communication characteristics of the real honey bees has proven its efficiency on solving different numerical and combinatorial optimization problems. In this study, the standard ABC algorithm and its well-known variants including the gbest-guided ABC algorithm, the differential evolution based ABC/best/1 and ABC/best/2 algorithms, crossover ABC algorithm, converge-onlookers ABC algorithm and quick ABC algorithm were assessed using the electroencephalographic signal decomposition based optimization problems introduced at the 2015 Congress on Evolutionary Computing Big Data Competition. The experimental studies on solving big data optimization problems showed that the phase-divided structure of the standard ABC algorithm still protects its advantageous sides when the candidate food sources or solutions are generated by referencing the global best solution in the onlooker bee phase.

A Hyper Meta-Heuristic Cascaded Support Vector Machines for Big Data Cyber-Security

At an incredible speed, cyber security evolves in the ever-changing setting of attacks. Organisation processing of information inward and outward is huge in quantity and determining a threat amidst of information is challengeable. Late discovery of such instance is standstill challenge of the meticulous process. Thence, detection of intrusion and its prevention are rising challenge in Big data factors. the information inundation generally incorporate the Big data terms to dataset. The majorly focused issues are industrial oriented in big data challenge. Existing systems for big data cyber security problems are based on Online Support Vector Machines (OSVMs) framework. Bi-objective optimisation problem with primary objectives is designed as OSVMs configuration process for improving accuracy and less complexity of model. Here, a bi-objective optimization is implemented based on an Artificial Bee Colony (ABC). However, Online Support Vector Machines (OSVMs) has issue with computational complexity, and prematurity and local optimum is major problems in ABC algorithm. By overcoming this issue, developed research system designs an Ensemble Support Vector Machine (ESVM) framework for big data cyber security. Initially, the feature selection is done by using improved K-means clustering. Based on the selected features the intrusion detection and malware detection are performed using ESVM approach. In this proposed research work, a bi-objective optimization problem is designed as the ESVM configuration process for improving accuracy and less complexity of model and achieve its objectives. Cuckoo Search (CS) optimization algorithm is implemented for the bi-objective optimization. accuracy, precision, recall and f-measure are the parametric meters compared in proposed research attaining higher performance against existing approaches.

Parallelism Strategies for Big Data Delayed Transfer Entropy Evaluation

Generated and collected data have been rising with the popularization of technologies such as Internet of Things, social media, and smartphone, leading big data term creation. One class of big data hidden information is causality. Among the tools to infer causal relationships, there is Delay Transfer Entropy (DTE); however, it has a high demanding processing power. Many approaches were proposed to overcome DTE performance issues such as GPU and FPGA implementations. Our study compared different parallel strategies to calculate DTE from big data series using a heterogeneous Beowulf cluster. Task Parallelism was significantly faster in comparison to Data Parallelism. With big data trend in sight, these results may enable bigger datasets analysis or better statistical evidence.

Big Data Analysis Using Apache Spark MLlib and Hadoop HDFS with Scala and Java

Nowadays with the technology revolution the term of big data is a phenomenon of the decade moreover, it has a significant impact on our applied science trends. Exploring well big data tool is a necessary demand presently. Hadoop is a good big data analyzing technology, but it is slow because the Job result among each phase must be stored before the following phase is started as well as to the replication delays. Apache Spark is another tool that developed and established to be the real model for analyzing big data with its innovative processing framework inside the memory and high-level programming libraries for machine learning, efficient data treating and etc. In this paper, some comparisons are presented about the time performance evaluation among Scala and Java in apache spark MLlib. Many tests have been done in supervised and unsupervised machine learning methods with utilizing big datasets. However, loading the datasets from Hadoop HDFS as well as to the local disk to identify the pros and cons of each manner and discovering perfect reading or loading dataset situation to reach best execution style. The results showed that the performance of Scala about 10% to 20% is better than Java depending on the algorithm type. The aim of the study is to analyze big data with more suitable programming languages and as consequences gaining better performance.