Algorithms For Big Data Analytics Research Articles

Applied Mathematics in the Design of Efficient Algorithms for Big Data Analytics

Applied Mathematics in the Design of Efficient Algorithms for Big Data Analytics

Big Data Analytics Algorithms for Dynamic Pricing: The Legal Analysis of the Indonesia Competitions Law readiness in Digital Era

This article analyzes the readiness of Indonesian competition law regarding the utilization of big data and reinforcement learning as tools to improve retailers' pricing strategies, ultimately leading to increased profitability and higher customer engagement and loyalty. It conducts a comprehensive review of scholarly literature pertaining to adaptive algorithmic pricing, with a specific focus on analyzing trends and the impacts of algorithmic pricing strategies. The literature review spans the years 2018 to 2022 and adheres to PRISMA criteria, with academic journals from Scopus serving as the primary source of research papers. The findings of this review indicate that it is evident that the most frequently utilized type of algorithm is RL, that shares a resemblance to human learning processes. Competition law enforcement should consider the possibility of illicit agreements between these artificial agents of colluding companies. In light of the capacity of EAs to facilitate the coordination of illicit agreement, it is imperative to consider the reformulation of Article 5 of Law Number 5 of 1999 and Regulation of KPPU Number 4 of 2011, particularly regarding the classification of price-fixing, to be adjusted to the latest developments, particularly regarding "EAs". ITE Law also should not be limited to EAs merely acting as “tools”. Instead, it should acknowledge their capacity to function as “AI Agents” capable of autonomous action.

Deep learning from latent spatiotemporal information of the heart: Identifying advanced bioimaging markers from echocardiograms.

A key strength of echocardiography lies in its integration of comprehensive spatiotemporal cardiac imaging data in real-time, to aid frontline or bedside patient risk stratification and management. Nonetheless, its acquisition, processing, and interpretation are known to all be subject to heterogeneity from its reliance on manual and subjective human tracings, which challenges workflow and protocol standardization and final interpretation accuracy. In the era of advanced computational power, utilization of machine learning algorithms for big data analytics in echocardiography promises reduction in cost, cognitive errors, and intra- and inter-observer variability. Novel spatiotemporal deep learning (DL) models allow the integration of temporal arm information based on unlabeled pixel echocardiographic data for convolution of an adaptive semantic spatiotemporal calibration to construct personalized 4D heart meshes, assess global and regional cardiac function, detect early valve pathology, and differentiate uncommon cardiovascular disorders. Meanwhile, data visualization on spatiotemporal DL prediction models helps extract latent temporal imaging features to develop advanced imaging biomarkers in early disease stages and advance our understanding of pathophysiology to support the development of personalized prevention or treatment strategies. Since portable echocardiograms have been increasingly used as point-of-care imaging tools to aid rural care delivery, the application of these new spatiotemporal DL techniques show the potentials in streamlining echocardiographic acquisition, processing, and data analysis to improve workflow standardization and efficiencies, and provide risk stratification and decision supporting tools in real-time, to prompt the building of new imaging diagnostic networks to enhance rural healthcare engagement.

A Multidimensional Data Utility Evaluation and Pricing Scheme in the Big Data Market

Big data as a derivative of information technology facilitates the birth of data trading. The technology surrounding the business value of big data has come into focus. However, most of the current research focuses on improving the performance of big data analytics algorithms. Data pricing is still one of the main issues in data trading. Therefore, we aim to tackle the problem of evaluating the utility of data in the big data trading market and the problem of maximizing the profits of the various roles involved in data trading. To this end, we propose a Multidimensional Data Utility Evaluation (MDDUE) method through three data quality dimensions, namely, data size, availability, and completeness. Next, we propose a big data trading market model including data providers, service providers, and service users. An optimal data-pricing scheme based on a three-party Stackelberg game is proposed to maximize the participants’ profits. Finally, a machine learning model is used to verify the rationality and validity of the MDDUE. The results show that MDDUE can evaluate the utility of data more accurately than previous work. The existence and uniqueness of the Nash equilibrium are demonstrated through numerical experiments.

Credit Card-Not-Present Fraud Detection and Prevention Using Big Data Analytics Algorithms

Currently, fraud detection is employed in numerous domains, including banking, finance, insurance, government organizations, law enforcement, and so on. The amount of fraud attempts has recently grown significantly, making fraud detection critical when it comes to protecting your personal information or sensitive data. There are several forms of fraud issues, such as stolen credit cards, forged checks, deceptive accounting practices, card-not-present fraud (CNP), and so on. This article introduces the credit card-not-present fraud detection and prevention (CCFDP) method for dealing with CNP fraud utilizing big data analytics. In order to deal with suspicious behavior, the proposed CCFDP includes two steps: the fraud detection Process (FDP) and the fraud prevention process (FPP). The FDP examines the system to detect harmful behavior, after which the FPP assists in preventing malicious activity. Five cutting-edge methods are used in the FDP step: random undersampling (RU), t-distributed stochastic neighbor embedding (t-SNE), principal component analysis (PCA), singular value decomposition (SVD), and logistic regression learning (LRL). For conducting experiments, the FDP needs to balance the dataset. In order to overcome this issue, Random Undersampling is used. Furthermore, in order to better data presentation, FDP must lower the dimensionality characteristics. This procedure employs the t-SNE, PCA, and SVD algorithms, resulting in a speedier data training process and improved accuracy. The logistic regression learning (LRL) model is used by the FPP to evaluate the success and failure probability of CNP fraud. Python is used to implement the suggested CCFDP mechanism. We validate the efficacy of the hypothesized CCFDP mechanism based on the testing results.

Predictive Analytics Executed through the Use of Social Big Data and Machine Learning: An Imperious Result

Instability in important socioeconomic indicators can have far-reaching effects on global development. This thesis offers a set of one-of-a-kind big data analytics algorithms that operate on unstructured Web data streams to automatically infer events, knowledge graphs, and predictive models, allowing for a better understanding, definition, and anticipation of the volatility of socioeconomic indicators. This paper we presents four major results that expand previous knowledge. Given a large volume of diverse unstructured news streams, we first describe novel models for collecting events and learning spatio-temporal features of events from news streams. We explore two different kinds of event models: one that is based on the concept of event triggers, and another that is probabilistic and learns a generic class of meta-events by extracting named entities from text streams. The second piece of work investigates the challenge of gleaning knowledge graphs from time-sensitive data like news and events as they happen. Event graphs produce a condensed depiction of a chronology of events pertinent to a news query by characterizing linkages between them using "event-phenomenon graphs," while spatio-temporal article graphs capture innate links between news stories. In this paper we present the various result outcome for predictive result analysis.

Exploring the Path to Big Data Analytics Success in Health Care

Like Oxygen, the world is surrounded by data today. The quantity of data that we harvest and eat up is thriving aggressively in the digitized world. Increasing use of new innovations and social media generate vast amount of data that can earn splendid information if properly analyzed. This large dataset generally known as big data, do not fit in traditional databases because of its rich size. Big Data is a collection of data that is huge in volume, yet growing exponentially with time. It is a data with so large size and complexity that none of traditional data management tools can store it or process it efficiently. Organizations need to manage and analyze big data for better decision making and outcomes. So, big data analytics is receiving a great deal of attention today. In healthcare, big data analytics has the possibility of advanced patient care and clinical decision support. In this paper, we review the background and the various methods of big data analytics in healthcare. This paper also elaborates various platforms and algorithms for big data analytics and discussion on its advantages and challenges. This survey winds up with a discussion of challenges and future directions.

A new adaptive sampling algorithm for big data classification

The exponential growth of the quantity of data that circulates on the web led to the emergence of the big data phenomenon. This fact is a natural consequence of the proliferation of social media, mobile devices, the abundance of free online storage, and new technologies like the internet of things. Subsequently, big data has created several challenges to the computer science community, among which the large size of data is the most challenging. Traditional machine learning algorithms used mostly for insight extraction find themselves inadequate, even on high-performance computer architectures. For instance, big data analytics algorithms can overcome the size issue by either: (1) adapting the existing machine learning techniques to the scale of the big data; or, (2) by sampling big datasets, choosing randomly much smaller subsets of the data population, to meet what current algorithms can handle. In the present work, we aim to proceed through the second alternative to address the size challenge in the big data context. We propose intelligent sampling techniques based on Scalable Simple Random Sampling (ScaSRS) and Subsampled Double Bootstrap (SDB). Test results carried out on public generic datasets show that our proposal is able to address the size dimension efficiently. The proposed algorithms were evaluative on a classification task where the obtained results provided significant improvement compared to the state-of-the-art.

A two-level formal model for Big Data processing programs

• A high-level model for iterative and non-iterative large-scale data processing programs. • Formalizing data flow with Petri Nets and data processing operations with Monoid Algebra. • Application of the model to formalize mutation operators for mutation testing in data flow programs. • Proposal of new mutation operators for iterative data flow programs. This paper proposes a model for specifying data flow-based parallel data processing programs agnostic of target Big Data processing frameworks. The paper focuses on the formal abstract specification of non-iterative and iterative programs, generalizing the strategies adopted by data flow Big Data processing frameworks. The proposed model relies on Monoid Algebra and Petri Nets to abstract Big Data processing programs in two levels: a higher level representing the program data flow and a lower level representing data transformation operations (e.g., filtering, aggregation, join). We extend the model for data processing programs proposed in [1] , for modeling iterative data processing programs. The general specification of these programs implemented by data flow-based parallel programming models is essential given the democratization of iterative and greedy Big Data analytics algorithms. Indeed, these algorithms call for revisiting parallel programming models to express iterations. The paper gives a comparative analysis of the iteration strategies proposed by Apache Spark, DryadLINQ, Apache Beam, and Apache Flink. It discusses how the model achieves to generalize these strategies.

Local Re-Encoding for Coded Matrix Multiplication

Matrix multiplication is a fundamental operation in various algorithms for big data analytics and machine learning. As the size of the dataset increases rapidly, it is now a common practice to distribute the computation on multiple servers. As straggling servers are inevitable in a distributed infrastructure, various coding schemes have been proposed to tolerate potential stragglers. However, as resources are shared with other jobs in a distributed infrastructure and their performance can change dynamically, the optimal way of encoding the input matrices is not static. So far, all existing coding schemes require encoding input matrices in advance, and cannot change the coding schemes or adjust their parameters flexibly. In this paper, we propose a framework that can change the coding schemes and/or their parameters by locally re-encoding the coded task on each server. We first present this framework for entangled polynomial codes, which changes the coding parameters with marginal overhead and saves job completion time. We then extend the framework for matrices with bounded entries, achieving a higher level of flexibility for local re-encoding while maintaining better numerical stability.

Machine Learning-based Decision-Making Systems, Cloud Computing and Blockchain Technologies, and Big Data Analytics Algorithms in Accounting and Auditing

Machine Learning-based Decision-Making Systems, Cloud Computing and Blockchain Technologies, and Big Data Analytics Algorithms in Accounting and Auditing

Multi Objective Optimization Based Feature Selection Algorithms for Big Data Analytics: A Review

Dimension reduction or feature selection is thought to be the backbone of big data applications in order to improve performance. Many scholars have shifted their attention in recent years to data science and analysis for real-time applications using big data integration. It takes a long time for humans to interact with big data. As a result, while handling high workload in a distributed system, it is necessary to make feature selection elastic and scalable. In this study, a survey of alternative optimizing techniques for feature selection are presented, as well as an analytical result analysis of their limits. This study contributes to the development of a method for improving the efficiency of feature selection in big complicated data sets.

Multi Objective Optimization Based Feature Selection Algorithms for Big Data Analytics: A Review

Dimension reduction or feature selection is thought to be the backbone of big data applications in order to improve performance. Many scholars have shifted their attention in recent years to data science and analysis for real-time applications using big data integration. It takes a long time for humans to interact with big data. As a result, while handling high workload in a distributed system, it is necessary to make feature selection elastic and scalable. In this study, a survey of alternative optimizing techniques for feature selection are presented, as well as an analytical result analysis of their limits. This study contributes to the development of a method for improving the efficiency of feature selection in big complicated data sets.

A Clustering Hybrid Algorithm for Smart Datasets using Machine Learning

In the field of data science, Machine Learning is treated as sub-field which primarily deals with designing of algorithms which have ability to learn from previous information and make future predictions accordingly. In traditional computational world the Machine Learning was generally performed on highly performance servers and machines. The implementation of these concepts on Big Data analytics algorithms has high potential and is still in its early stages. So far as machine learning is concerned, performance measure is an important parameter to evaluate the overall functionality of the algorithms. The data set is a different entity and the measuring of performance on a data which is unseen is also called as test set, and training set is a Data set which is training itself. The Data Mining is extensively using learning algorithms for data analysis and to formulate future predications based on archived data. The research presented provides a step forward to make smart data sets out of training data set by evaluating machine learning algorithm. The research presented a novel hybrid algorithm that attempts to incorporate the feature of similarities in Random Forest machine learning algorithm for improving the classification accuracy and efficiency of working.

On using MapReduce to scale algorithms for Big Data analytics: a case study

IntroductionMany data analytics algorithms are originally designed for in-memory data. Parallel and distributed computing is a natural first remedy to scale these algorithms to “Big algorithms” for large-scale data. Advances in many Big Data analytics algorithms are contributed by MapReduce, a programming paradigm that enables parallel and distributed execution of massive data processing on large clusters of machines. Much research has focused on building efficient naive MapReduce-based algorithms or extending MapReduce mechanisms to enhance performance. However, we argue that these should not be the only research directions to pursue. We conjecture that when naive MapReduce-based solutions do not perform well, it could be because certain classes of algorithms are not amendable to MapReduce model and one should find a fundamentally different approach to a new MapReduce-based solution.Case descriptionThis paper investigates a case study of a scaling problem of “Big algorithms” for a popular association rule-mining algorithm, particularly the development of Apriori algorithm in MapReduce model.Discussion and evaluationFormal and empirical illustrations are explored to compare our proposed MapReduce-based Apriori algorithm with previous solutions. The findings support our conjecture and our study shows promising results compared to the state-of-the-art performer with 7% increase in performance on the average of transactions ranging from 10,000 to 120,000.ConclusionsThe results confirm that effective MapReduce implementation should avoid dependent iterations, such as that of the original sequential Apriori algorithm. These findings could lead to many more alternative non-naive MapReduce-based “Big algorithms”.

A data structure perspective to the RDD-based Apriori algorithm on Spark

During the recent years, a number of efficient and scalable frequent itemset mining algorithms for big data analytics have been proposed by many researchers. Initially, MapReduce-based frequent itemset mining algorithms on Hadoop cluster were proposed. Although, Hadoop has been developed as a cluster computing system for handling and processing big data, but the performance of Hadoop does not meet the expectation for the iterative algorithms of data mining, due to its high I/O, and writing and then reading intermediate results in the disk. Consequently, Spark has been developed as another cluster computing infrastructure which is much faster than Hadoop due to its in-memory computation. It is highly suitable for iterative algorithms and supports batch, interactive, iterative, and stream processing of data. Many frequent itemset mining algorithms have been re-designed on the Spark, and most of them are Apriori-based. All these Spark-based Apriori algorithms use Hash Tree as the underlying data structure. This paper investigates the efficiency of various data structures for the Spark-based Apriori. Although, the data structure perspective has been investigated previously, but for MapReduce-based Apriori, and it must be re-investigated in the distributed computing environment of Spark. The considered underlying data structures are Hash Tree, Trie, and Hash Table Trie. The experimental results on the benchmark datasets show that the performance of Spark-based Apriori with Trie and Hash Table Trie are almost similar but both perform many times better than Hash Tree in the distributed computing environment of Spark.

A General Perspective of Big Data Analytics Algorithms, Tools and Techniques

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

A Data Structure Perspective to the RDD-based Apriori Algorithm on Spark

During the recent years, a number of efficient and scalable frequent itemset mining algorithms for big data analytics have been proposed by many researchers. Initially, MapReduce-based frequent itemset mining algorithms on Hadoop cluster were proposed. Although, Hadoop has been developed as a cluster computing system for handling and processing big data, but the performance of Hadoop does not meet the expectation for the iterative algorithms of data mining, due to its high I/O, and writing and then reading intermediate results in the disk. Consequently, Spark has been developed as another cluster computing infrastructure which is much faster than Hadoop due to its in-memory computation. It is highly suitable for iterative algorithms and supports batch, interactive, iterative, and stream processing of data. Many frequent itemset mining algorithms have been re-designed on the Spark, and most of them are Apriori-based. All these Spark-based Apriori algorithms use Hash Tree as the underlying data structure. This paper investigates the efficiency of various data structures for the Spark-based Apriori. Although, the data structure perspective has been investigated previously, but for MapReduce-based Apriori, and it must be re-investigated in the distributed computing environment of Spark. The considered underlying data structures are Hash Tree, Trie, and Hash Table Trie. The experimental results on the benchmark datasets show that the performance of Spark-based Apriori with Trie and Hash Table Trie are almost similar but both perform many times better than Hash Tree in the distributed computing environment of Spark.

Data Stream Partitioning Algorithms for Big Data Analytics: A Review

Data Stream Partitioning Algorithms for Big Data Analytics: A Review

Analysing real world data streams with spatio-temporal correlations: Entropy vs. Pearson correlation

Smart Cities use different Internet of Things (IoT) data sources and rely on big data analytics to obtain information or extract actionable knowledge crucial for urban planners for efficiently use and plan the construction infrastructures. Big data analytics algorithms often consider the correlation of different patterns and various data types. However, the use of different techniques to measure the correlation with smart cities data and the exploitation of correlations to infer new knowledge are still open questions. This paper proposes a methodology to analyse data streams, based on spatio-temporal correlations using different correlation algorithms and provides a discussion on co-occurrence vs. causation. The proposed method is evaluated using traffic data collected from the road sensors in the city of Aarhus in Denmark.