Transactional Data Streams Research Articles

A Geth-based real-time detection system for sandwich attacks in Ethereum

With the rapid development of the Ethereum ecosystem and the increasing applications of decentralized finance (DeFi), the security research of smart contracts and blockchain transactions has attracted more and more attention. In particular, front-running attacks on the Ethereum platform have become a major security concern. These attack strategies exploit the transparency and certainty of the blockchain, enabling attackers to gain unfair economic benefits by manipulating the transaction order. This study proposes a sandwich attack detection system integrated into the go-Ethereum client (Geth). This system, by analyzing transaction data streams, effectively detects and defends against front-running and sandwich attacks. It achieves real-time analysis of transactions within blocks, quickly and effectively identifying abnormal patterns and potential attack behaviors. The system has been optimized for performance, with an average processing time of 0.442 s per block and an accuracy rate of 83%. Response time for real-time detection new blocks is within 5 s, with the majority occurring between 1 and 2 s, which is considered acceptable. Research findings indicate that as a part of the go-Ethereum client, this detection system helps enhance the security of the Ethereum blockchain, contributing to the protection of DeFi users’ private funds and the safety of smart contracts. The primary contribution of this study lies in offering an efficient blockchain transaction monitoring system, capable of accurately detecting sandwich attack transactions within blocks while maintaining normal operation speeds as a full node.

Risk management opportunities in auto insurance: A focus on the value of vehicles

AbstractRatemaking and risk‐based pricing revolve around the value at risk, involving a tendency of higher values implying higher premiums. Traditionally, vehicle values are set at “list price” and depreciate over time on a one‐size‐fits‐all factor model. If values change over time, so should premiums, but a single factor table that only goes downward to flat is out of sorts with today. In today's transactional data streams, vehicle features, and values can be easily observed—of note, a shift in product mix that has more options and higher retained values creates a gap in actual versus traditionally expected insurance values. This gap most acutely appears at the vehicle “as built” level, which is more granular than current segmentation levels of insurance ratemaking vehicle valuation analysis. The result is leaving companies to simply raise base rates universally until they can adapt more flexible price‐to‐value methods at an “as built” price accuracy sophistication level.

FePARM: The Frequency-Patterned Associative Rule Mining Framework on Consumer Purchasing-Pattern for Online Shops

1Malasowe, Bridget Ogheneovo,, 2Okpako, Abugor Ejaita, 3 , 4Ejeh, Patrick Ogholuwarami, 5Ojugo, Arnold Adimabua & 6 1Department of Computer Science, University of Delta, Agbor, Nigeria 2Department of Cyber Security, University of Delta, Agbor, Nigeria 3Department of Cyber Security, Faculty of Computing, Delta State University of Science and Technology, Ozoro. Nigeria. 4Department of Computer Science, Dennis Osadebay University, Anwai-Asaba, Nigeria 5 Department of Computer Science, Federal University of Petroleum Resources Effurun, Nigeria 6 Department of Computer Science, Federal University of Petroleum Resources Effurun, Nigeria E-mails: 1bridget.malasowe@unidel.edu.ng; 2ejaita.okpako@unidel.edu.ng; 3okpormd@dsust.edu.ng, 4 patrick.ejeh@dou.edu.ng; 5ojugo.arnold@fupre.edu.ng, 6ako.rita@fupre.edu.ng ABSTRACT Transaction data often is a true presentation of consumers’ buying behavior, stored as a set of relational records, which properly harnessed via mining – can help businesses improve their sales volume as a decision support system. Managing such a system can pose many issues to biz such as feature evolution, concept evolution, concept drift, and infinite data length – and often makes it impractical to effectively store such big-data. To curb this, previous studies have assumed data to be stationary in using associative rule mining. This has deprived such systems of the flexibility and adaptiveness required to handle the dynamics of concept drift that characterizes transaction datasets. Our study thus proposes a basket frequent pattern growth trained associative rule mining model to handle large data. The dataset was retrieved from the Delta-Mall Asaba and consists of 556,000 transaction consumer records. The model consists of six-layers, and yields the best result with a 0.1 value for both the confidence and support level(s) at a 94% accuracy, sensitivity of 87%, and a specificity of 32% with a 20-second convergence and processing time. Keywords: Big-Data Analysis, Feature Evolution, Inventory Model, Market Basket Analysis, Businesses, Transactional Data Streams, Concept Drift CISDI Journal Reference Format Malasowe, B.O., Okpako, A.E., Okpor, M.D., Ejeh, P.O., Ojugo, A.A. & Ako, R.E. (2024): FePARM: The Frequency-Patterned Associative Rule Mining Framework on Consumer Purchasing-Pattern for Online Shops. Computing, Information Systems, Development Informatics & Allied Research Journal. Vol 15 No 2, Pp 15-28. dx.doi.org/10.22624/AIMS/CISDI/V15N2P2-1. Available online at www.isteams.net/cisdijournal

FePARM: The Frequency-Patterned Associative Rule Mining Framework on Consumer Purchasing-Pattern for Online Shops

Transaction data often is a true presentation of consumers’ buying behavior, stored as a set of relational records, which properly harnessed via mining – can help businesses improve their sales volume as a decision support system. Managing such a system can pose many issues to biz such as feature evolution, concept evolution, concept drift, and infinite data length – and often makes it impractical to effectively store such big-data. To curb this, previous studies have assumed data to be stationary in using associative rule mining. This has deprived such systems of the flexibility and adaptiveness required to handle the dynamics of concept drift that characterizes transaction datasets. Our study thus proposes a basket frequent pattern growth trained associative rule mining model to handle large data. The dataset was retrieved from the Delta-Mall Asaba and consists of 556,000 transaction consumer records. The model consists of six-layers, and yields the best result with a 0.1 value for both the confidence and support level(s) at a 94% accuracy, sensitivity of 87%, and a specificity of 32% with a 20-second convergence and processing time. Keywords: Big-Data Analysis, Feature Evolution, Inventory Model, Market Basket Analysis, Businesses, Transactional Data Streams, Concept Drift CISDI Journal Reference Format Malasowe, B.O., Okpako, E.A., Ashioba, N.O., Ejeh, P.O., Ojugo, A.A. & Ako, R.E. (2024): FePARM: The Frequency-Patterned Associative Rule Mining Framework on Consumer Purchasing-Pattern for Online Shops. Computing, Information Systems, Development Informatics & Allied Research Journal. Vol 15 No 2, Pp 1-14. dx.doi.org/10.22624/AIMS/CISDI/V15N2P2. Available online at www.isteams.net/cisdijournal

Anytime Frequent Itemset Mining of Transactional Data Streams

Mining frequent itemsets from transactional data streams has become very essential in today's world with many applications such as stock market analysis, retail chain analysis, web log analysis, etc. Various algorithms have been proposed to efficiently mine single-port and multi-port transactional streams within the constraints of limited time and memory. However, all of them are budget algorithms, i.e., they are not capable of handling varying inter-arrival rate of transactions and high speed streams. They are constrained by a maximum limit to the inter-arrival rate of transactions, beyond which they fail to process. Also, these algorithms are not capable of giving immediate mining results, even with compromised accuracy if required. The above two properties characterize an anytime algorithm. We propose AnyFI, which is the first anytime frequent itemset mining algorithm for data streams. AnyFI uses a novel data structure - BFI-forest, which is capable of handling transactions arriving at variable rate. It maintains itemsets in BFI-forest in such a way that it can give a mining result almost immediately when the time allowance to mine is very less and can refine its accuracy with increase in time allowance. We also propose MPAnyFI which extends AnyFI into a parallel framework for anytime frequent itemset mining of multi-port data streams over commodity clusters. It uses AnyFI at each computing node of the cluster. Our extensive experimental analysis shows that AnyFI can handle high stream speeds close to 60,000 trans/sec with recall close to 100%. They also show the efficiency of MPAnyFI.

Effective approximation of parametrized closure systems over transactional data streams

Strongly closed itemsets, defined by a parameterized closure operator, are a generalization of ordinary closed itemsets. Depending on the strength of closedness, the family of strongly closed itemsets typically forms a tiny subfamily of ordinary closed itemsets that is stable against changes in the input. In this paper we consider the problem of mining strongly closed itemsets from transactional data streams. Utilizing their algebraic and algorithmic properties, we propose an algorithm based on reservoir sampling for approximating this type of itemsets in the landmark streaming setting, prove its correctness, and show empirically that it yields a considerable speed-up over a straightforward naive algorithm without any significant loss in precision and recall. We motivate the problem setting considered by two practical applications. In particular, we first experimentally demonstrate that the above properties, i.e., compactness and stability, make strongly closed itemsets an excellent indicator of certain types of concept drifts in transactional data streams. As a second application we consider computer-aided product configuration, a real-world problem raised by an industrial project. For this problem, which is essentially exact concept identification, we propose a learning algorithm based on a certain type of subset queries formed by strongly closed itemsets and show on real-world datasets that it requires significantly less query evaluations than a naive algorithm based on membership queries.

Scalable Data Analytics Market Basket Model for Transactional Data Streams

Transactional data streams (TDS) are incremental in nature thus, the process of mining is complicated. Such complications arise from challenges such as infinite length, feature evolution, concept evolution and concept drift. Tracking concept drift challenge is very difficult, thus very important for Market Basket Analysis (MBA) applications. Hence, the need for a strategy to accurately determine the suitability of item pairs within the available billions of pairs to solve concept drift chalenge of TDS in MBA. In this work, a Scalable Data Analytics Market Basket Model (SDAMBM) that handles concept drift issues in MBA was developed. Transactional data of 1,112,000 were extracted from a grocery store using Extraction, Transformation and Loading approach and 556,000 instances of the data were simulated from a cloud database. Calibev function was used to caliberate the data nodes. Lugui 7.2.9 and Comprehensive R Archive Network were used for table pivoting between the simulated data and the data collected. The SDAMBM was developed using a combination of components from elixir big data architecture, the research conceptual model and consumer behavior theories. Toad Modeler was then used to assemble the model. The SDAMBM was implemented using Monarch and Tableau to generate insights and data visualization of the transactions. Intelligent interpreters for auto decision grid, selectivity mechanism and customer insights were used as metrics to evaluate the model. The result showed that 79% of the customers from the customers’ consumption pattern of the SDAMBM preferred buying snacks and drink as shown in the visualization report through the SDAMBM visualization dashboard. Finally, this study provided a data analytics approach for managing concept drift challenge in customers’ buying pattern. Furthermore, a distinctive model for managing concept drift was also achieved. It is therefore recommended that the SDAMBM should be adopted for the enhancement of customers buying and consumption pattern by business ventures, organizations and retailers.

A three-phase approach to differentially private crucial patterns mining over data streams

Frequent patterns mining over transactional data streams is an important task for a wide range of online data mining applications. Nevertheless, mining crucial patterns is even more appropriate than frequent patterns over transactional data streams, because crucial patterns are the subset of frequent patterns with the minimum storage cost and information lossless extraction. In this paper, we argue that the privacy of mining crucial patterns from data streams (i.e., aggregating information from individuals) is more likely to be leaked than static scenarios, due to successive releases. However, to the best of our knowledge, there is little work on differential privacy in continuously publishing crucial patterns from data streams. To this end, this paper proposes a real-time differentially private crucial pattern computation algorithm which designs a three-phase mechanism (i.e., the preprocessing phase, the deep-going calculation phase, and the noise-mining phase) at every timestamp. The algorithm is able to not only improve the utility of the crucial pattern statistics as much as possible which satisfy differential privacy, but also reduce the average mining time without incurring high maintenance cost according to the feature of crucial patterns. To reduce the number of calls to crucial pattern computation algorithm, we design two-dissimilarity formulas according to the relationship between frequent patterns and crucial patterns to decide to return either low noisy statistic or accurately approximated statistic in the first two phases. When the low noisy statistic needs to be turned, the algorithm goes into the noise-mining phase. To obtain private crucial patterns, we first filter crucial pattern candidate set by perturbing the scoring functions, and then add independent Laplace noise to their supports. Finally, we conduct extensive experiments on dense datasets and sparse datasets to show the effectiveness and efficiency of our algorithm.

High utility drift detection in quantitative data streams

This paper presents an efficient algorithm for detecting changes (drifts) in the utility distributions of patterns, named High Utility Drift Detection in Transactional Data Stream (HUDD-TDS). The algorithm is specifically suitable for quantitative data streams, where each item has a unit profit, and non-binary purchase quantities are allowed. We propose a method that enables the HUDD-TDS algorithm to be used in an online setting to detect drifts. An important property of HUDD-TDS is that it can quickly adapt to changes in streams, while considering older transactions to be less important than new ones. Furthermore, the proposed method applies statistical testing based on Hoeffding bound with Bonferroni correction in order to ensure that only significant changes are reported to the user. This test allows identifying a change (drift) if the difference between current and the previous time window is significant in terms of utility distribution. In this work, we focus on both local and global utility drifts. A local utility drift is a drift in the utility distribution of a single pattern, whereas a global utility drift is a change in the utilities of all high utility itemsets. In order to be able to compute the similarity of different high utility itemsets to detect drifts, we propose a new distance measure function. The results of our experiments on both real world and synthetic datasets show the feasibility and efficiency of the proposed HUDD-TDS algorithm.

Adapting to Concept Drift in Credit Card Transaction Data Streams Using Contextual Bandits and Decision Trees

Credit card transactions predicted to be fraudulent by automated detection systems are typically handed over to human experts for verification. To limit costs, it is standard practice to select only the most suspicious transactions for investigation. We claim that a trade-off between exploration and exploitation is imperative to enable adaptation to changes in behavior (concept drift). Exploration consists of the selection and investigation of transactions with the purpose of improving predictive models, and exploitation consists of investigating transactions detected to be suspicious. Modeling the detection of fraudulent transactions as rewarding, we use an incremental Regression Tree learner to create clusters of transactions with similar expected rewards. This enables the use of a Contextual Multi-Armed Bandit (CMAB) algorithm to provide the exploration/exploitation trade-off. We introduce a novel variant of a CMAB algorithm that makes use of the structure of this tree, and use Semi-Supervised Learning to grow the tree using unlabeled data. The approach is evaluated on a real dataset and data generated by a simulator that adds concept drift by adapting the behavior of fraudsters to avoid detection. It outperforms frequently used offline models in terms of cumulative rewards, in particular in the presence of concept drift.

Two Privacy-Preserving Approaches for Publishing Transactional Data Streams

Recently, data mining over transactional data streams has become an attractive research area. However, releasing raw transactional data streams, in which only explicit identifying information must be removed, may compromise individual privacy. Many privacy-preserving approaches have been proposed for publishing static transactional data. Due to the characteristics of data streams, which must be processed quickly, static data anonymization methods cannot be directly applied to data streams. In this paper, we first analyze the privacy problem in publishing transactional data streams based on a sliding window. Then, we present two dynamic algorithms with generalization and suppression to anonymize continuously a sliding window to make it satisfy $\rho $ -uncertainty by structuring an affected sensitive rules trie, because the removal and addition of transactions may make the current sliding window fail to satisfy $\rho $ -uncertainty. Experimental results show that our methods are more efficient than sliding window anonymization with batch processing by using existing static anonymization methods.

Mining Frequent Item Sets in Asynchronous Transactional Data Streams over Time Sensitive Sliding Windows Model

EPs (Extracting Frequent Patterns) from the continuous transactional data streams is a challenging and critical task in some of the applications, such as web mining, data analysis and retail market, prediction and network monitoring, or analysis of stock market exchange data. Many algorithms have been developed previously for mining FPs (Frequent Patterns) from a data stream. Such algorithms are currently highly required to develop new solutions and approaches to the precise handling of data streams. New techniques, solutions, or approaches are developed to address unbounded, ordered, and continuous sequences of data and for the generation of data at a rapid speed from data streams. Hence, extracting FPs using fresh or recent data involves the high-level analysis of data streams. We have suggested an efficient technique for the window sliding model; this technique extracts new and fresh FPs from high-speed data streams. In this study, a CPILT (Compacted Tree Compact Pattern Tree) is developed to capture the latest contents in the stream and to efficiently remove outdated contents from the data stream. The main concept introduced in this work on CPILT is the dynamic restructuring of a tree, which is helpful in producing a compacted tree and the frequency descending structure of a tree on runtime. With the help of the mining technique of FP growth, a complete list of new and fresh FPs is obtained from a CPILT using an existing window. The memory usage and time complexity of the latest FPs in high-speed data streams can efficiently be determined through proper experimentation and analysis.

Mining Low, Medium and High Profit Customers Over Transactional Data Stream

In Frequent Itemset Mining each item in transaction is represented by a binary value means 1 for present and 0 for absent. But There are several other parameter are also important like quantity, price or and profit of each item. Quantity, price or and profit these parameter are important in retail markets to find high utility itemset. High utility item set are those items which have utility value larger than a user specified value of minimum utility. The basic meaning of utility is the profitability of items to the users. However, quantity is significant for addressing real world decision problems that require maximizing the utility in an organization. For making business customer relationship management in retail markets customer base is the prime objective [17,20]. Data mining techniques are nowadays used to predict buying behavior of customers by analyzing transactional data. This paper introduces effective customer classification in retail marketing by using transactional utility value. This helps the business man to find those customers which contribute maximum profit to the overall transaction scenario

Sliding Window-based Frequent Itemsets Mining over Data Streams using Tail Pointer Table

AbstractMining frequent itemsets over transaction data streams is critical for many applications, such as wireless sensor networks, analysis of retail market data, and stock market predication. The sliding window method is an important way of mining frequent itemsets over data streams. The speed of the sliding window is affected not only by the efficiency of the mining algorithm, but also by the efficiency of updating data. In this paper, we propose a new data structure with a Tail Pointer Table and a corresponding mining algorithm; we also propose a algorithm COFI2, a revised version of the frequent itemsets mining algorithm COFI (Co-Occurrence Frequent-Item), to reduce the temporal and memory requirements. Further, theoretical analysis and experiments are carried out to prove their effectiveness.

UT-Tree: Efficient mining of high utility itemsets from data streams

High utility itemsets mining is a hot topic in data stream mining. It is essential that the mining algorithm should be efficient in both time and space for data stream is continuous and unbounded. To the best of our knowledge, the existing algorithms require multiple database scans to mine high utility itemsets, and this hinders their efficiency. In this paper, we propose a new data structure, called UT-Tree Utility on Tail Tree, for maintaining utility information of transaction itemsets to avoid multiple database scans. The UT-Tree is created with one database scan, and contains a fixed number of transaction itemsets; utility information is stored on tail-nodes only. Based on the proposed data structure and the sliding window approach, we propose a mining algorithm, called HUM-UT High Utility itemsets Mining based on UT-Tree, to find high utility itemsets from transactional data streams. The HUM-UT algorithm mines high utility itemsets from the UT-Tree without additional database scan. Experiment results show that our algorithm has better performance and is more stable under different experimental conditions than the state-of-the-art algorithm HUPMS in terms of time and space.

Non-homogeneous Poisson Model for Mining Frequency of an Item from Data Stream

A data stream is essentially a virtually unbounded sequence of data items arriving at a rapid rate. Mining frequent patterns from the stream of data is a difficult task. This paper use non-homogeneous Poisson process to find the frequency of an item set from transactional data streams. We develop models by using mean value function from Goel–Okumoto model. The concept of split Poisson process and Bayesian model are used for developing a model for the prediction of number of arrivals of an item with in a particular time period.

Mining frequent patterns in a varying-size sliding window of online transactional data streams

In some data stream applications, the information embedded in the data arriving in the most recent time period is of particular interest. This paper proposes a method for efficiently mining the frequent patterns in a varying-size sliding window of online data streams. To highlight recent frequent patterns in the data stream, a time decay model is used to differentiate the patterns of recently generated transactions from historical transactions. The derived concrete bounds of the decay factor can achieve either 100% recall or 100% precision. A summary data structure, named SWP-tree, is proposed for capturing the content of the transactions in the sliding window by scanning the stream only once. In order to speed up online processing of new transactions, the information of frequent patterns recorded in the SWP-tree is updated in an incrementally way. To make the mining operation efficient, the SWP-tree is periodically pruned by identifying insignificant patterns, which include two kinds of obsolete pattern and two kinds of infrequent pattern. Since the sliding window can change its size, the effect of window size is examined. The performance of the proposed technique is evaluated via simulation experiments. The results show that the proposed method is both efficient and scalable, and that it outperforms comparable algorithms.

Content Based Sampling over Transactional Data Streams

Content Based Sampling over Transactional Data Streams

EstMax: Tracing Maximal Frequent Item Sets Instantly over Online Transactional Data Streams

Frequent item set mining is one of the most challenging issues for descriptive data mining. In general, its resulting set tends to produce a large number of frequent item sets. To represent them in a more compact notation, closed or maximal frequent item sets are often used but finding such item sets over online transactional data streams is not easy due to the requirements of a data stream. For this purpose, this paper proposes a method of tracing the set of MFIs instantly over an online data stream. The method, namely estMax, maintains the set of frequent item sets by a prefix tree and extracts all MFIs without any additional superset/subset checking mechanism. Upon processing a new transaction, those frequent item sets that are matched maximally by the transaction are newly marked in their corresponding nodes of the prefix tree as candidates for MFIs. At the same time, if any subset of a newly marked item set has been already marked as a candidate MFI by a previous transaction, it is cleared as well. By employing this additional step, it is possible to extract the set of MFIs at any moment. The performance of the estMax method is comparatively analyzed by a series of experiments to identify its various characteristics.

Discovering frequent itemsets over transactional data streams through an efficient and stable approximate approach

A data stream is a massive and unbounded sequence of data elements that are continuously generated at a fast speed. Compared with traditional approaches, data mining in data streams is more challenging since several extra requirements need to be satisfied. In this paper, we propose a mining algorithm for finding frequent itemsets over the transactional data stream. Unlike most of existing algorithms, our method works based on the theory of Approximate Inclusion–Exclusion. Without incrementally maintaining the overall synopsis of the stream, we can approximate the itemsets’ counts according to certain kept information and the counts bounding technique. Some additional techniques are designed and integrated into the algorithm for performance improvement. Besides, the performance of the proposed algorithm is tested and analyzed through a series of experiments.