Advent Of Big Data Research Articles

Construction And Performance Evaluation of Big Data Prediction Model Based on Fuzzy Clustering Algorithm in Cloud Computing Environment

In the evolving landscape of biomedical biometrics, where multimodal approaches are increasingly crucial for reliable user authentication, this research presents a comprehensive study. The primary focus is on the construction and performance evaluation of a robust big data prediction model within a cloud computing environment. The advent of big data and cloud computing has revolutionized the field of biomedical biometrics, offering immense potential for advanced data analysis and prediction. This research presents the development and evaluation of a robust prediction model for multimodal biometric data in biomedical applications. The proposed model incorporation of Reliable Discrete Variable Topology (RDVT) into the prediction model. RDVT introduces a topological data structure that enhances data reliability and ensures the integrity of multimodal biometric information. The construction and training of the prediction model are meticulously detailed, encompassing data preprocessing, feature extraction, clustering, classification, and model evaluation. Additionally, the integration of a fuzzy clustering algorithm enhances the model's ability to handle uncertainty and imprecision in biometric data. The advancement of multimodal biometrics in the biomedical field by introducing the Reliable Discrete Variable Topology (RDVT) and a big data prediction model based on a fuzzy clustering algorithm in a cloud computing environment. The model's performance is rigorously assessed through extensive experimentation, including accuracy, precision, recall, and F1-score measurements.

The Application of Big Data in Economic Statistics

The advent of big data has revolutionized the field of economic statistics, providing researchers and policymakers with vast amounts of accurate, timely, and diverse data. At the beginning, this review provides the foundations of big data in economic statistics from two parts-- the definition of big data and the characteristics of big data. Then, this comprehensive literature review aims to explore the applications of big data in economic statistics from five aspects: unit operation management, human resource management, fund operation analysis, risk management, and allocation of internal resources. Additionally, the review analyzes the three advantages and three challenges associated with the use of big data in economic statistics. Besides, the review also provides a number of solutions to improve current problems of economics statistics, including updating statistical concept, cultivating professional statistical talents, and increasing equipment investment. By understanding the potential and limitations of big data, policymakers can make informed decisions to promote economic growth and development.

MolToxPred: small molecule toxicity prediction using machine learning approach.

Different types of chemicals and products may exhibit various health risks when administered into the human body. For toxicity reasons, the number of new drugs entering the market through the conventional drug development process has been reduced over the years. However, with the advent of big data and artificial intelligence, machine learning techniques have emerged as a potential solution for predicting toxicity and ensuring efficient drug development and chemical safety. An ML model for toxicity prediction can reduce experimental costs and time while addressing ethical concerns by drastically reducing the need for animals and clinical trials. Herein, MolToxPred, an ML-based tool, has been developed using a stacked model approach to predict the potential toxicity of small molecules and metabolites. The stacked model consists of random forest, multi-layer perceptron, and LightGBM as base classifiers and Logistic Regression as the meta classifier. For training and validation purposes, a comprehensive set of toxic and non-toxic molecules is curated. Different structural and physicochemical-based features in the form of molecular descriptors and fingerprints were employed. MolToxPred utilizes a comprehensive feature selection process and optimizes its hyperparameters through Bayesian optimization with stratified 5-fold cross-validation. In the evaluation phase, MolToxPred achieved an AUROC of 87.76% on the test set and 88.84% on an external validation set. The McNemar test was used as the post-hoc test to determine if the stacked models' performance was significantly different compared to the base learners. The developed stacked model outperformed its base classifiers and an existing tool in the literature, reaffirming its better performance. The hypothesis is that the incorporation of a diverse set of data, the subsequent feature selection, and a stacked ensemble approach give MolToxPred the edge over other methods. In addition to this, an attempt has been made to identify structural alerts responsible for endpoints of the Tox21 data to determine the association of a molecule with a plausible downstream pathway of action. MolToxPred may be helpful for drug discovery and regulatory pipelines in pharmaceutical and other industries for in silico toxicity prediction of small molecule candidates.

A Model Driven Framework for Collaborative and Dynamic Design and Implementation of NoSQL-Oriented Data warehouses

Nowadays, modernizing the data warehouse ecosystem is a key challenge in decision support systems. This modernization is crucial for ensuring scalability and meeting evolving business requirements, especially with the advent of big data. A promising solution involves implementing data warehouses with contemporary data stores, such as NoSQL. In this context, we introduce in this paper a framework that leverages Model-Driven Architecture (MDA) to design and implement modern data warehouses across NoSQL data stores. Our MDA approach aims to offer a collaborative, dynamic, and reusable process for developing NoSQL-oriented data warehouses tailored to specific project requirements. It facilitates the automatic and dynamic generation of a hybrid data warehouse model from its conceptual model, which encompasses structural, domain, and access parameters. Moreover, our framework includes the generation of implementation code for the data warehouse, along with a set of files to validate, document, and illustrate the data warehouse schema on a target platform. Finally, we present a detailed case study to highlight the effectiveness of our MDA framework.

Database profile of the Fukuoka-City Information Platform for Community-based Integrated Care.

The Fukuoka-City Information Platform for Community-based Integrated Care is an advanced big data platform that aggregates information on the health and medical services of Fukuoka citizens. Fukuoka City is engaged in a joint project with Kyushu University to promote policy making through a large-scale real-world data analysis. This paper describes the framework for this cooperative effort and the features of the analytical platform. Fukuoka City is the fifth most populous ordinance-designated city in Japan, with an estimated population of approximately 1.6 million. Under an agreement with Fukuoka City, Kyushu University was granted access to a portion of the city's anonymized healthcare database as secondary-use information. The database contains information on resident registration, health insurance claims, specific health checkups and health checkups for the older adults, specific health guidance, long-term care insurance data, and cancer screenings collected after fiscal year 2012. Each of these constituent datasets can be interlinked using anonymized hashed key variables, allowing individuals to be followed across databases and over time. The platform allows longitudinal investigation of the complex association between various aspects of healthcare, such as medical procedures, examinations, interviews, medical costs, long-term care certifications, and care costs. The platform can provide valuable public-health information because it is relatively large for a single database, and because it allows analysis of data across multiple domains and tracing of individuals over time.

Data-Driven Decision-Making: Leveraging Analytics for Product Management

In the rapidly evolving landscape of contemporary business, the role of product management has become increasingly complex and multifaceted (1). With the advent of big data and advanced analytics, product managers are presented with unprecedented opportunities to enhance decision-making processes through the integration of data-driven insights (2). This scholarly article delves into the pivotal intersection of data analytics and product management, exploring the transformative potential of leveraging data-driven decision-making strategies. The article begins by elucidating the foundational principles of data-driven decision-making and its relevance to the discipline of product management (7). It investigates the evolving nature of consumer behavior and market dynamics, emphasizing the critical need for product managers to assimilate and interpret vast datasets. By elucidating the symbiotic relationship between analytics and product management, this research seeks to provide a comprehensive understanding of how data-driven insights can empower decision-makers to anticipate market trends, identify consumer preferences, and optimize product development strategies. Furthermore, the article delves into the methodologies and tools employed in contemporary data analytics, elucidating their applicability to the product management domain (4). From predictive modeling to machine learning algorithms, an array of analytical techniques is explored, each offering unique insights into consumer behavior, product performance, and market dynamics (8). By providing a nuanced analysis of these methodologies, this research equips product managers with the knowledge necessary to navigate the complexities of data integration and interpretation. The article also scrutinizes the challenges and ethical considerations associated with data-driven decision-making in the realm of product management (3). Recognizing the importance of responsible data usage, the research discusses strategies for mitigating biases, ensuring data privacy, and fostering transparency in decision-making processes. Ultimately, this scholarly exploration aims to contribute to the evolving discourse on product management by elucidating the transformative potential of data-driven decision-making (5). By synthesizing theoretical frameworks, practical insights, and real-world case studies, the article serves as a valuable resource for academics, practitioners, and industry leaders seeking to harness the power of analytics to drive innovation, enhance competitiveness, and navigate the complexities of the modern business landscape (6)..

Yield Predict: A Crop Yield Prediction Framework for Smart Farms

In today’s world the most important thing for living in the Indian economy is agriculture.In recent years, machine learning approaches are gaining popularity with the advent of big data. Adoption of these approaches in the agricultural sector has immense potential to increase crop productivity and quality. As we know, India’s economy primarily depends on agriculture. For successful production of crop we must ensure whether a particular crop will yield in particular soil and weather condition Machine learning (ML) plays a significant role as it has decision support tool for Crop Yield Prediction (CYP) including supporting decisions on what crops to grow and what to do during the growing season of the crops. This script makes novel by the usage of simple parameters like State, district, season area and the user can predict the yield of the crop in which year he or she wants to.

Digital Ethics for Biometric Applications in a Smart City

From border control using fingerprints to law enforcement with video surveillance to self-activating devices via voice identification, biometric data is used in many applications in the contemporary context of a Smart City. Biometric data consists of human characteristics that can identify one person from others. Given the advent of big data and the ability to collect large amounts of data about people, data sources ranging from fingerprints to typing patterns can build an identifying profile of a person. In this article, we examine different types of biometric data used in a smart city based on a framework that differentiates between profile initialization and identification processes. Then, we discuss digital ethics within the usage of biometric data along the lines of data permissibility and renewability. Finally, we provide suggestions for improving biometric data collection and processing in the modern smart city.

Big Data Management: Empowering Sustainable Logistics with Data-Driven Operation Optimization

Numerous disciplines, including logistics, have experienced a paradigm shift since the advent of big data. As the backbone of many industries, logistics has been significantly impacted by the expansion of big data. The purpose of this paper is to investigate the far-reaching effects of big data technology on the development of sustainable logistics, with a particular emphasis on the role of big data in various parts of the logistics process. Through a literature review and case analysis, this paper focuses on summarizing the application of big data in all aspects of logistics, and selecting and analyzing the experiences and lessons of businesses that have successfully implemented big data logistics practices. It finds that by optimizing the operation model and other means, big data effectively improved logistics efficiency. Simultaneously, intelligent solutions supported by big data can also aid in reducing the impact on the environment, for instance by encouraging the use of alternative energy vehicles. In addition, big data can encourage various entities to engage in collaborative innovation and promote the continuous generation of logistics-related solutions.

Revealing social dimensions of urban mobility with big data: A timely dialogue

Considered a total social phenomenon, mobility is at the center of intricate social dynamics in cities and serves as a reading lens to understand the whole society. With the advent of big data, the potential for using mobility as a key social analyzer was unleashed in the past decade. The purpose of this research is to systematically review the evolution of big data's role in revealing social dimensions of urban mobility and discuss how they have contributed to various research domains from early 2010s to now. Six major research topics are detected from the selected online academic corpuses by conducting keywords-driven topic modeling techniques, reflecting diverse research interests in networked mobilities, human dynamics in spaces, event modeling, spatial underpinnings, travel behaviors and mobility patterns, and sociodemographic heterogeneity. The six topics reveal a comprehensive, research-interests, evolution pattern, and present current trends on using big data to uncover social dimensions of human mobility activities. Given these observations, we contend that big data has two contributions to revealing social dimensions of urban mobility: as an efficiency advancement and as an equity lens. Furthermore, the possible limitations and potential opportunities of big data applications in the existing scholarship are discussed. The review is intended to serve as a timely retrospective of societal-focused mobility studies, as well as a starting point for various stakeholders to collectively contribute to a desirable future in terms of mobility.

Optimizing Big Data Analytics for Reliability and Resilience: A Survey of Techniques and Applications

The advent of big data has revolutionized various industries, enabling organizations to make data- driven decisions and gain valuable insights. However, the sheer volume, velocity, and variety of big data pose significant challenges in ensuring the reliability and resilience of big data analytics pipelines. In this context, optimization techniques play a crucial role in enhancing the reliability and resilience of big data analytics. This paper provides a comprehensive survey of optimization techniques for reliable and resilient big data analytics. The paper first discusses the motivation for optimizing big data analytics for reliability and resilience. Then, it presents a detailed overview of various optimization techniques, including resource optimization, data partitioning, data compression, load balancing, and fault detection and tolerance. Finally, the paper discusses the applications of optimization techniques in various big data analytics domains, such as real-time analytics, fraud detection, recommendation systems, predictive analytics, and risk management.

Advances, challenges, and future research needs in machine learning-based crash prediction models: A systematic review

Accurately modelling crashes, and predicting crash occurrence and associated severities are a prerequisite for devising countermeasures and developing effective road safety management strategies. To this end, crash prediction modelling using machine learning has evolved over two decades. With the advent of big data that provides unprecedented opportunities to better understand the crash mechanism and its determinants, such efforts will likely be accelerated. To gear these efforts, understanding state-of-the-art machine learning-based crash prediction models becomes paramount to summarise the lessons learned from past efforts, which can assist in developing robust and accurate models. This review paper aims to address this gap by systematically reviewing the machine learning studies on crash modelling. Models are reviewed from three aspects of the application: (a) crash occurrence (or real-time crash) prediction, (b) crash frequency prediction, and (c) injury severity prediction. Further, model intricacies that impact model performance are identified and thoroughly reviewed. This comprehensive review highlights specific gaps and future research needs in three aforementioned model applications, such as improper selection of non-crash events for crash occurrence models, the inability of future forecasting of crash frequency models, and inconsistency in injury severity classes. Critical research needs relating to model development, evaluation, and application are also discussed. This review envisages methodological advancements in machine learning models for crash prediction modelling and leveraging big data to better link crashes with its determinants.

Big data in insurance contracts - a tool for good, or bad?

Big data is changing the way many companies conduct business on a day-to-day basis. Insurers are notorious for utilising data in the risk assessment of prospective and current policyholders. The use of such risk assessment mechanisms in insurance has resulted in some discrimination between various policyholders but this has been held to be justifiable due to the fact that it is based on actuarial science and is therefore viewed as fair. However, the advent of big data, data analytics, algorithms, and artificial intelligence is providing insurers with far more sophisticated data about potential policyholders. This may prove to be beneficial to insurers in many ways, but it also brings about additional possibilities of exclusion within the industry. This use of big data has the potential to increase social injustices within our country, which is something that needs to be avoided as much as possible. Discrimination, through the use of big data, is a reality that needs to be addressed by insurers and regulators alike. Achieving social justice as far as possible in the insurance industry is crucial and also requires considerations in the areas of morality and ethics. The reason for this is that the very nature of big data is integrally linked to the assessment of the policyholder's moral risks and hazards for the benefit of the insurer is often linked to personal circumstances and, sometimes, the financial circumstances of the policyholder, and may even speak to the policyholder's integrity. Although potentially beneficial for the insurers from a risk assessment perspective, the use of big data has ethical and moral considerations within the insurance context. After all, the insurance industry's collection, storage, and use of big data raises ethical and moral concerns and casts a shadow on the manner it is used to assess policyholders. This discussion highlights the need for regulatory oversight of big data, an aspect that is notably missing in the South African legislative framework.

The Transformation of International Marketing Strategy under the Background of Big Data

The emergence of big data has brought many opportunities to all walks of life, but it also faces enormous challenges. In order to improve the competitive power of international trade, companies need to actively respond to the development trend of large data, the advantage of the characteristics of big data, raising the level of competition in international trade, improve the international marketing strategy, promote the all-round development of international trade and industry. With the technical support of big data, we can dig deep into data information, gain insight into consumers' buying behavior, and inject innovative power into the marketing of enterprises. Combining big data technology with a marketing strategy can also help enterprise marketers to adapt to the market development trend, collect data, analyze data, reduce risks, and enhance the quality of enterprise development. The advent of big data has brought about significant changes in the field of international marketing, forcing companies to adapt their marketing strategies to suit the new realities. This paper examines the challenges faced by international trade in the era of big data and explores how companies can transform their international marketing strategies to take advantage of the opportunities presented by this new technology. The paper concludes that companies that are able to harness the power of big data will be better positioned to compete in the global marketplace.

Ethical Framework for Multi-Source Big Data Analytics Disease Surveillance in Public Healthcare

Big data analytics can revolutionize disease surveillance by integrating multiple data sources and analyzing those diverse data sources, allowing for early monitoring, detection, and response to disease outbreaks. Since the growth in technology, together with the advent of big data, there is growing interest in exploring the potential of multiple data sources for disease surveillance. However, there are still ethical concerns when it comes to big data usage for disease surveillance. Hence, this study uses of scoping review with the assistance of socio-technical systems theory, together with social contact theory to examine the ethical dimensions of employing big data analytics from diverse sources for disease surveillance, while emphasizing the importance of maintaining ethical standards in public healthcare practices. The study discovered that informed consent, privacy, and confidentiality are still major challenges when it comes to the integration of big data sources in disease surveillance. The study then recommends a unified ethical framework for enhanced disease surveillance in public healthcare through Multi-Data Source big data Analytics.

Leveraging Analytics and Big Data for Informed Decision-Making: A Comprehensive Review

Abstract–The advent of Big Data and advanced analytics technologies has revolutionized the way organizations collect, process, and utilize data to drive decision-making. This research paper provides a comprehensive review of the role of analytics and Big Data in contemporary business, highlighting their significance, applications, challenges, and future prospects. By examining various case studies and industry examples, we aim to illustrate how organizations across different sectors harness the power of analytics and Big Data to gain competitive advantages and address complex challenges. The paper also discusses the ethical and privacy considerations associated with the utilization of large-scale data analytics. Keywords: Big Data, Analytics, Decision-Making, Data Privacy, Data Warehousing, Business Intelligence, Predictive Analytics.

Lambda Architecture-Based Big Data System for Large-Scale Targeted Social Engineering Email Detection

In this research, we delve deep into the realm of Targeted Social Engineering Email Detection, presenting a novel approach that harnesses the power of Lambda Architecture (LA). Our innovative methodology strategically segments the BERT model into two distinct components: the embedding generator and the classification segment. This segmentation not only optimizes resource consumption but also improves system efficiency, making it a pioneering step in the field. Our empirical findings, derived from a rigorous comparison between the fastText and BERT models, underscore the superior performance of the latter. Specifically, The BERT model has high precision rates for identifying malicious and benign emails, with impressive recall values and F1 scores. Its overall accuracy rate was 0.9988, with a Matthews Correlation Coefficient value of 0.9978. In comparison, the fastText model showed lower precision rates. Leveraging principles reminiscent of the Lambda architecture, our study delves into the performance dynamics of data processing models. The Separated-BERT (Sep-BERT) model emerges as a robust contender, adept at managing both real-time (stream) and large-scale (batch) data processing. Compared to the traditional BERT, Sep-BERT showcased superior efficiency, with reduced memory and CPU consumption across diverse email sizes and ingestion rates. This efficiency, combined with rapid inference times, positions Sep-BERT as a scalable and cost-effective solution, aligning well with the demands of Lambda- inspired architectures. This study marks a significant step forward in the fields of big data and cybersecurity. By introducing a novel methodology and demonstrating its efficacy in detecting targeted social engineering emails, we not only advance the state of knowledge in these domains but also lay a robust foundation for future research endeavors, emphasizing the transformative potential of integrating advanced big data frameworks with machine learning models.

Ecolo-zip: A global, rich and granular characterization of biogeophysical ecology for 1.5 million postal codes

The inescapable fact that human life is perpetually embedded in a tangible biogeophysical environment – and the consequences that this has for individuals and societies – have long fascinated scholars of all backgrounds. Technological progress and the advent of big data have spurred ever-more precise attempts to quantify our biogeophysical environments. However, many such datasets lack spatial granularity, global coverage, content depth, or accessibility. Here, we introduce ecolo-zip, a novel geospatial dataset that provides a granular-yet-global, parsimonious-yet-rich ecological characterization of over 1.5 million postal codes across 94 countries and regions. Combining two large-scale satellite image resources (ASTER; SRTM, ICC = 0.999) and a customized geospatial sampling model, we provide high-resolution indicators of physical topography (elevation, mountainousness, distance to sea), vegetation (normalized difference vegetation index), and climate (surface temperature). With this resource – featuring methodological details, visualizations, and application suggestions – we hope to contribute towards understanding the multi-faceted interactions between humans and their environments.

Supply Chain Management Optimization Based on Bigdata Analysis

Contemporarily, in the context of big data, enterprises need to continuously optimize and innovate the supply chain, change the traditional supply chain management process pattern as well as optimization the whole supply processes. These actions are able to strengthen the actual effect level of supply chain process operation and achieve more benefits for the enterprise. Under the background of big data, more information management technologies are widely used, and supply chain management needs to clarify the deficiencies in its own operation, and reengineer and adjust the deficiencies. This paper will systematically discusses the implementation of the state-of-art bigdata analysis techniques in SCM. According to the analysis and evaluations of the measures, the usage of advanced big data technology will help increase the cultivation and establishment of management talent resources, so as to optimize the supply chain management process. Overall, these results shed light on guiding further exploration of supply chain management optimization.

Context-aware Big Data Quality Assessment: A Scoping Review

The term data quality refers to measuring the fitness of data regarding the intended usage. Poor data quality leads to inadequate, inconsistent, and erroneous decisions that could escalate the computational cost, cause a decline in profits, and cause customer churn. Thus, data quality is crucial for researchers and industry practitioners. Different factors drive the assessment of data quality. Data context is deemed one of the key factors due to the contextual diversity of real-world use cases of various entities such as people and organizations. Data used in a specific context (e.g., an organization policy) may need to be more efficacious for another context. Hence, implementing a data quality assessment solution in different contexts is challenging. Traditional technologies for data quality assessment reached the pinnacle of maturity. Existing solutions can solve most of the quality issues. The data context in these solutions is defined as validation rules applied within the ETL (extract, transform, load) process, i.e., the data warehousing process. In contrast to traditional data quality management, it is impossible to specify all the data semantics beforehand for big data. We need context-aware data quality rules to detect semantic errors in a massive amount of heterogeneous data generated at high speed. While many researchers tackle the quality issues of big data, they define the data context from a specific standpoint. Although data quality is a longstanding research issue in academia and industries, it remains an open issue, especially with the advent of big data, which has fostered the challenge of data quality assessment more than ever. This article provides a scoping review to study the existing context-aware data quality assessment solutions, starting with the existing big data quality solutions in general and then covering context-aware solutions. The strength and weaknesses of such solutions are outlined and discussed. The survey showed that none of the existing data quality assessment solutions could guarantee context awareness with the ability to handle big data. Notably, each solution dealt only with a partial view of the context. We compared the existing quality models and solutions to reach a comprehensive view covering the aspects of context awareness when assessing data quality. This led us to a set of recommendations framed in a methodological framework shaping the design and implementation of any context-aware data quality service for big data. Open challenges are then identified and discussed.