Big Data Life Cycle Research Articles

Strengthening big data security and privacy in libraries: A Review of ISO standards and real-world cyberattack case studies

This paper presents a review of ISO standards and real-world case studies to provide a robust framework for enhancing big data security and privacy in libraries. The study synthesises relevant ISO standards such as ISO/IEC 27001, ISO/IEC 27014, and ISO/IEC TS 19608 and analyses library cyberattacks to identify vulnerabilities and security gaps. The paper provides practical recommendations and mitigation strategies particularly beneficial for librarians in resource-limited regions and developing countries. By integrating these standards with case study insights and addressing security requirements across the big data lifecycle, the paper offers an actionable guide to help libraries strengthen their cybersecurity frameworks, improve risk management, and maintain public trust.

The credibility assessment of Twitter/X users based organization objectives by heterogeneous resources in big data life cycle

Social network data, such as Twitter/X, is of Big Social Data type. Big social data describes people's social behaviors and interactions. They have high business value for decision-making in organizations. However, because of the anonymous nature of social network users, their credibility is ambiguous. Credibility expresses the accuracy and value of big social data. Despite extensive research on the credibility of big social data, most methods have not paid sufficient attention to the important dimensions of their assessment, including user expertise based on topic, selecting social network features, and labeling them. Furthermore, these methods cannot manage the time, high volume, and speed of big social data. To address these issues, this paper presents a novel model for assessing the credibility of Twitter/X users by integrating Twitter/X with Google Scholar. The model automatically defines users' credibility labels using Google Scholar. Machine learning feature selection methods also select features that affect the credibility of Twitter/X users based on the topic. This study uses Google Scholar and the BerTopic algorithm for effective topic modeling on Twitter/X. The model considers unrelated data management, dynamic user credibility, and organizing activities based on the Big Data lifecycle. Finally, using Linear Regression, Support Vector Regression, K-Nearest Neighbor, Random Forest, Classification and Regression Trees algorithms, the model predicts the credibility of Twitter/X users and proves that it performed better than similar models through Classification and Regression Trees. In addition, the model is generalizable for all organizational purposes due to the integration of heterogeneous resources and feature selection methods.

A systematic data characteristic understanding framework towards physical-sensor big data challenges

Big data present new opportunities for modern society while posing challenges for data scientists. Recent advancements in sensor networks and the widespread adoption of the Internet of Things (IoT) have led to the collection of physical-sensor data on an enormous scale. However, significant challenges arise in conducting high-quality data analytics within the realm of big data. To uncover big data challenges and enhance data quality, it is essential to quantitatively unveil data characteristics. Furthermore, the existing studies lack analysis of the specific time-related characteristics of physical-sensor data. Enhancing the efficiency and precision of data analytics through the big data lifecycle requires a comprehensive understanding of data characteristics to address the hidden big data challenges. To fill in the research gap, this paper proposes a systematic data characteristic framework based on a 6Vs model. The framework aims to unveil the data characteristics in terms of data volume, variety, velocity, veracity, value, and variability through a set of statistical indicators. This model improves the objectivity of data characteristic understanding by relying solely on data-driven indicators. The indicators related to time-related characteristics in physical-sensor data are also included for the analysis of temporal aspects in the physical-sensor data. Furthermore, the big data challenges are linked to each dimension of the 6Vs model to gain a quantitative understanding of the data challenges. Finally, a pipeline is developed to implement the proposed framework, and two case studies are conducted to illustrate the process of understanding the physical-sensor data characteristics and making recommendations for data preprocessing to address the big data challenges. The proposed framework is able to analyze the characteristics of all physical-sensor data, therefore, identifying potential challenges in subsequent analytics, and providing recommendations for data preprocessing. Furthermore, the characteristic indicators can be used to analyze other types of big data.

Privacy Challenges and Solutions in Big Data Analytics: A Comprehensive Review

Abstract: Large data collections with a more intricate and diversified structure are referred to as "big data." These traits are typically associated with more challenges when it comes to storing, analyzing, and applying additional procedures or getting results. The technique of analyzing vast quantities of intricate data to uncover patterns or connections that are concealed is known as "big data analytics." Nonetheless, there is a clear conflict between big data's extensive use and its security and privacy. This paper examines privacy needs in big data, distinguishes between privacy and security, and focuses on privacy and security issues in large data. This study addresses the applications of privacy in business by utilizing established techniques like HybrEx, k-anonymity, T-closeness, and L-diversity. Many privacy-preserving techniques have been created for protection of privacy at various phases of the big data life cycle (such as data generation, storage, and processing). This work aims to explain the issues facing current privacy preservation strategies in big data and to give a comprehensive evaluation of them. This study also introduces new approaches to privacy preservation in big data, such as quick anonymization of massive data streams, identity-based anonymization, differential privacy, privacy preserving big data publishing, and hiding a needle in a haystack. This study discusses big data privacy and security issues in healthcare. A comparative analysis of some contemporary large data privacy approaches is also conducted. A comparative analysis of the latest big data privacy approaches is also conducted.

Tackling security and privacy challenges in the realm of big data analytics

As organizations increasingly harness the power of big data analytics to derive insights and drive decision-making, the paramount concerns of security and privacy have come to the forefront. This paper presents a comprehensive framework for addressing the multifaceted challenges of big data security and privacy. Drawing on a synthesis of cutting-edge technologies, encryption methods, and access control mechanisms, our approach aims to fortify the entire big data lifecycle. The paper delves into innovative strategies for secure data storage, transmission, and processing, ensuring that sensitive information is shielded from unauthorized access or malicious attacks. Additionally, the framework incorporates robust privacy-preserving techniques, including anonymization and differential privacy, to uphold individual confidentiality. Through a meticulous analysis of current trends, emerging threats, and regulatory landscapes, this paper not only provides theoretical insights but also practical guidelines for organizations seeking to navigate the intricate landscape of big data while safeguarding the integrity, security, and privacy of the vast datasets at their disposal.

Big Data Life Cycle in Shop-Floor–Trends and Challenges

Big data is defined as a large set of data that could be structured or unstructured. In manufacturing shop-floor, big data incorporates data collected at every stage of the production process. This includes data from machines, connecting devices, and even manufacturing operators. The large size of the data available on the manufacturing shop-floor presents a need for the establishment of tools and techniques along with associated best practices to leverage the advantage of data-driven performance improvement and optimization. There also exists a need for a better understanding of the approaches and techniques at various stages of the data life cycle. In the work carried out, the data life-cycle in shop-floor is studied with a focus on each of the components - Data sources, collection, transmission, storage, processing, and visualization. A narrative literature review driven by two research questions is provided to study trends and challenges in the field. The selection of papers is supported by an analysis of n-grams. Those are used to comprehensively characterize the main technological and methodological aspects and as starting point to discuss potential future research directions. A detailed review of the current trends in different data life cycle stages is provided. In the end, the discussion of the existing challenges is also presented.

A Big Data Approach to Explore Medical Imaging Repositories Based on DICOM

Medical imaging repositories based on the DICOM format such as Picture Archiving and Communication Systems have a huge potential from a big data perspective. The study reported by this paper aimed to verify on how the big data lifecycle processes, from production to consumption might be implemented to take advantage of the DICOM standard and respective data sources. After identifying the different processes (i.e., data collection, integration, filtering, anonymization and enrichment, and knowledge extraction) the study demonstrate their implementation using an open-source application able to access the DICOM metadata independently of the medical imaging modalities and equipment manufacturers.

Research on the Construction of Urban Leisure Physical Culture Healthy Big Data Service Platform Based on In-Depth Learning

The national fitness service system must focus on meeting the sports needs of residents, innovate the construction mode of urban leisure sports big data service platform, and promote the construction of urban leisure sports health big data service platform. The healthy development of urban leisure sports should also adhere to the principles of demand orientation and problem awareness. Big data is an industry with urban leisure and sports health as its core data. It is a production process that continues to cycle in the life cycle of big data. Therefore, this paper uses in-depth learning to study the construction of big data service platform for urban leisure sports health. The in-depth learning improvement project is an expert team organized by the urban leisure sports health basic textbook development center of the Ministry of education. The data revolution driven by deep learning examined the value of data, conquered the data ocean, and created a new era of personalized medicine in the fields of urban leisure, sports, and health. On the basis of in-depth learning, this paper constructs a big data service platform for urban leisure sports health. On the one hand, the establishment of this platform will help to speed up the construction process of sports service informatization and the reform of various fields of sports, on the other hand, it will help to promote the improvement of the physical quality of the whole people.

WITHDRAWN: Real-Time Monitoring Of Big Data Sports Teaching Data Based On Complex Embedded System

The growing mindset that oversees one of the fundamental difficulties created in the big data austerity tings (population) internet industry. Unpredictable from many parts of the big data life cycle, for example, collecting, for example, cloud packing them and having the latest progress cycle information including, including, Extensible Markup Language (XML) and resource interpretation framework. This management structure that is going on simply because the way manage the system does not change. They have become a basic device in the world of games. Previously, the execution of the game to usually completed by cutting the hours or days after the game — this video has a detailed cycle to the original map with each player developing. The ever-evolving gaming application remote sensor and minor improvements give new doors open for improvement. Sensor gadgets are gradually becoming smarter and more flexible, adjusting to freely linked and approved execution information technology capabilities. The various advantages, organizational capability and usefulness of this phone will allow to use the program, then again, with impossible observations and schedule changes. This paper proposes a simple and generic game system to integrate into the online survey of imagination and standard input strategy to ensure non-exclusivity.

Federated Learning Approach to Protect Healthcare Data over Big Data Scenario

The benefits and drawbacks of various technologies, as well as the scope of their application, are thoroughly discussed. The use of anonymity technology and differential privacy in data collection can aid in the prevention of attacks based on background knowledge gleaned from data integration and fusion. The majority of medical big data are stored on a cloud computing platform during the storage stage. To ensure the confidentiality and integrity of the information stored, encryption and auditing procedures are frequently used. Access control mechanisms are mostly used during the data sharing stage to regulate the objects that have access to the data. The privacy protection of medical and health big data is carried out under the supervision of machine learning during the data analysis stage. Finally, acceptable ideas are put forward from the management level as a result of the general privacy protection concerns that exist throughout the life cycle of medical big data throughout the industry.

C4I System Security Architecture: A Perspective on Big Data Lifecycle in a Military Environment

Although the defense field is also one of the key areas that use big data for security reasons, there is a lack of study that designs system frameworks and presents security requirements to implement big data in defense. However, we overcome the security matters by examining the battlefield environment and the system through the flow of data in the battlefield. As such, this research was conducted to apply big data in the defense domain, which is a unique field. In particular, a three-layered system framework was designed to apply big data in the C4I system, which collects, manages, and analyzes data generated from the battlefield, and the security measures required for each layer were developed. First, to enhance the general understanding of big data and the military environment, an overview of the C4I system, the characteristics of the 6V’s, and the five-phase big data lifecycle were described. While presenting a framework that divides the C4I system into three layers, the roles and components of each layer are described in detail, considering the big data lifecycle and system framework. A security architecture is finally proposed by specifying security requirements for each field in the three-layered C4I system. The proposed system framework and security architecture more accurately explain the unique nature of the military domain than those studied in healthcare, smart grids, and smart cities; development directions requiring further research are described.

Development of Privacy-Preservation of Big Data with support of Hyperledger Fabric and IPFS

Alpesh Vaghela et al., International Journal of Advanced Trends in Computer Science and Engineering, 10(5), September - October 2021, 2930 – 2935 2930 ABSTRACT Academics and industry researchers alike find privacy-preservation of large data to be a very intriguing field of study. Data collection, storage, and processing are the three steps of big data's life cycle. At different stages of the big data life cycle, different privacy and security solutions are used. Many health-care stakeholders are working together to develop a new pattern for safeguarding people from an unknown disease while also promoting economic prosperity. The methods of big data processing and big data analytics will be employed to discover new economic growth patterns. Because the current method of data anonymization leads to data breaches, researchers needed to develop a new way of large data mining or knowledge discovery in databases (KDD), in which numerous parties share their data to identify new patterns. This study introduces a novel way for data mining privacy protection based on Blockchain and the InterPlanetary File System (IPFS) (PPDM). The authors propose leveraging Blockchain and IPFS to create the ChainPPDM approach for preserving big data privacy. The data saved on the blockchain is immutable, transparent, and safe, and it allows for decentralized storage. IPFS is a distributed file system that stores data in a decentralized manner.

Big data quality framework: a holistic approach to continuous quality management

Big Data is an essential research area for governments, institutions, and private agencies to support their analytics decisions. Big Data refers to all about data, how it is collected, processed, and analyzed to generate value-added data-driven insights and decisions. Degradation in Data Quality may result in unpredictable consequences. In this case, confidence and worthiness in the data and its source are lost. In the Big Data context, data characteristics, such as volume, multi-heterogeneous data sources, and fast data generation, increase the risk of quality degradation and require efficient mechanisms to check data worthiness. However, ensuring Big Data Quality (BDQ) is a very costly and time-consuming process, since excessive computing resources are required. Maintaining Quality through the Big Data lifecycle requires quality profiling and verification before its processing decision. A BDQ Management Framework for enhancing the pre-processing activities while strengthening data control is proposed. The proposed framework uses a new concept called Big Data Quality Profile. This concept captures quality outline, requirements, attributes, dimensions, scores, and rules. Using Big Data profiling and sampling components of the framework, a faster and efficient data quality estimation is initiated before and after an intermediate pre-processing phase. The exploratory profiling component of the framework plays an initial role in quality profiling; it uses a set of predefined quality metrics to evaluate important data quality dimensions. It generates quality rules by applying various pre-processing activities and their related functions. These rules mainly aim at the Data Quality Profile and result in quality scores for the selected quality attributes. The framework implementation and dataflow management across various quality management processes have been discussed, further some ongoing work on framework evaluation and deployment to support quality evaluation decisions conclude the paper.

PRIVACY-PRESERVING IN MULTI-PARTY DATA RELEASE ON CLOUD FOR BIG DATA USING FUSION LEARNING

The data volume increases every day and the next wave of apps cannot be envisaged without the created and executed data-driven algorithms. In this post, we undertook an extensive survey on privacy issues in the context of big data. At every point of the Big Data life cycle, we explored privacy challenges and discussed some of the advantages and disadvantages of the Big Data application of new privacy conservation schemes. Much progress has been made in protecting the protection of consumers from data production to data storage, but many transparent questions and hurdles exist.

Security and Privacy Risk Assessment of Energy Big Data in Cloud Environment.

Considering the importance of energy in our lives and its impact on other critical infrastructures, this paper starts from the whole life cycle of big data and divides the security and privacy risk factors of energy big data into five stages: data collection, data transmission, data storage, data use, and data destruction. Integrating into the consideration of cloud environment, this paper fully analyzes the risk factors of each stage and establishes a risk assessment index system for the security and privacy of energy big data. According to the different degrees of risk impact, AHP method is used to give indexes weights, genetic algorithm is used to optimize the initial weights and thresholds of BP neural network, and then the optimized weights and thresholds are given to BP neural network, and the evaluation samples in the database are used to train it. Then, the trained model is used to evaluate a case to verify the applicability of the model.

Security and Privacy in Big Data Life Cycle: A Survey and Open Challenges

The use of big data in various fields has led to a rapid increase in a wide variety of data resources, and various data analysis technologies such as standardized data mining and statistical analysis techniques are accelerating the continuous expansion of the big data market. An important characteristic of big data is that data from various sources have life cycles from collection to destruction, and new information can be derived through analysis, combination, and utilization. However, each phase of the life cycle presents data security and reliability issues, making the protection of personally identifiable information a critical objective. In particular, user tendencies can be analyzed using various big data analytics, and this information leads to the invasion of personal privacy. Therefore, this paper identifies threats and security issues that occur in the life cycle of big data by confirming the current standards developed by international standardization organizations and analyzing related studies. In addition, we divide a big data life cycle into five phases (i.e., collection, storage, analytics, utilization, and destruction), and define the security taxonomy of the big data life cycle based on the identified threats and security issues.

Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

Day by day advanced web technologies have led to tremendous growth amount of daily data generated volumes. This mountain of huge and spread data sets leads to phenomenon that called big data which is a collection of massive, heterogeneous, unstructured, enormous and complex data sets. Big Data life cycle could be represented as, Collecting (capture), storing, distribute, manipulating, interpreting, analyzing, investigate and visualizing big data. Traditional techniques as Relational Database Management System (RDBMS) couldn’t handle big data because it has its own limitations, so Advancement in computing architecture is required to handle both the data storage requisites and the weighty processing needed to analyze huge volumes and variety of data economically. There are many technologies manipulating a big data, one of them is hadoop. Hadoop could be understand as an open source spread data processing that is one of the prominent and well known solutions to overcome handling big data problem. Apache Hadoop was based on Google File System and Map Reduce programming paradigm. Through this paper we dived to search for all big data characteristics starting from first three V's that have been extended during time through researches to be more than fifty six V's and making comparisons between researchers to reach to best representation and the precise clarification of all big data V’s characteristics. We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed cloud based environment. This paper mainly focuses on different components of hadoop like Hive, Pig, and Hbase, etc. Also we institutes absolute description of Hadoop Pros and cons and improvements to face hadoop problems by choosing proposed Cost-efficient Scheduler Algorithm for heterogeneous Hadoop system.

A Cross-Domain Comparative Study of Big Data Architectures

Nowadays, a variety of Big Data architectures are emerging to organize the Big Data life cycle. While some of these architectures are proposed for general usage, many of them are proposed in a specific application domain such as smart cities, transportation, healthcare, and agriculture. There is, however, a lack of understanding of how and why Big Data architectures vary in different domains and how the Big Data architecture strategy in one domain may possibly advance other domains. Therefore, this paper surveys and compares the Big Data architectures in different application domains. It also chooses a representative architecture of each researched application domain to indicate which Big Data architecture from a given domain the researchers and practitioners may possibly start from. Next, a pairwise cross-domain comparison among the Big Data architectures is presented to outline the similarities and differences between the domain-specific architectures. Finally, the paper provides a set of practical guidelines for Big Data researchers and practitioners to build and improve Big Data architectures based on the knowledge gathered in this study.

Environmental Open Data in Urban Platforms: An Approach to the Big Data Life Cycle

ABSTRACTThrough a literature review, this paper proposes an assessment framework to evaluate environmental open data in urban platforms under the data life cycle approach. For this purpose, a set of quantitative and qualitative indicators has been defined for each of the stages of the data life cycle: acquisition, access, analysis, and application. As confirmation of the capabilities of this approach, we illustrate the results of its application in six data portals along with analyses, comparisons, and conclusions regarding the results. The cases reveal that an interest exists in providing environmental information, but the capability to transform the data information into knowledge relevant for cities is rather low.

Power of Big Data System for Storing and Processing Huge Data

Big data uses storage of huge data with some approaches and techniques to manage and process them. During the past few years the number of persons using internet, email and other internet-based applications has been growing tremendously. Big Data is mainly characterized by 3V’s (Volume, Velocity and, Variety). The Big Data Architecture Framework (BDAF) is proposed to address all aspects of the Big Data Ecosystem. BDAF includes components such as Big Data Infrastructure, Big Data Analytics, Data structures & models, Big Data Lifecycle Management and Big Data Security. Nowadays the volume of data used by the people throughout the world is increasing enormously and exponentially. So, the need for storing, processing and protecting large volume of data has been becoming a great challenge in the modern hyper-connected world. On the basis of work from home concept lot of software professionals are doing their jobs with their internet connected systems for development, implementation, testing and maintenance of various softwares. These professionals and experts are sending and receiving lot of data to various locations to their clients, higher authorities and other officials frequently depending upon their requirements. The traditional data management models are not efficient for today’s exponentially growing data from variety of industries. This challenging task of storing and managing huge volume of data is achieved in Big Data Systems. In this paper we try to give an overview of Big Data Analytics system for storing and processing huge volume of various types of data. Overwhelming the security threats due to various factors like viruses, worms, etc are also great challenges to protect huge volume of data in a big data system.