Large Transaction Databases Research Articles

Privacy-preserving association rule mining via multi-key fully homomorphic encryption

Association rule mining is an efficient method to mine the association relationships between different items from large transaction databases, but is vulnerable to privacy leakage as operates over users’ sensitive data directly. Privacy-preserving association rule mining has emerged to protect users’ privacy during rule mining. Unfortunately, existing privacy-preserving association rule mining schemes suffer from high overhead, fail to support multiple users, and are challenging to prevent collusion attacks between twin-server. To this end, in this paper, we propose a privacy-preserving association rule mining solution via multi-key fully homomorphic encryption over the torus (MKTFHE), which efficiently supports multiple users through a single server only. Specifically, we first construct some multi-key homomorphic gates based on MKTFHE. Then, we designed a series of privacy-preserving computational protocols based on multi-key homomorphic gates. Finally, we build a privacy-preserving association rule mining system with a single cloud server to support multiple users. Moreover, privacy analysis and performance evaluation demonstrate our proposal is efficient and feasible. In contrast to existing solutions, the proposed scheme outperforms encryption and communication, saving approximately 8.5% running time.

Analysis on the Establishment and Management of Library Resource Base Based on Modern Information Technology

The current rapid development of modern network information technology, along with the arrival of this trend, has profoundly changed people’s living and learning methods. In the past, the management of book resources in the traditional period has gradually exposed many drawbacks, and it is difficult to meet the technological development of the information age. Combined with the complexity of book management parameters, high variable dimensions, and the characteristics of multiattribute data points including not only numerical attributes but also category attributes and mixed attributes, the fuzzy clustering algorithm is combined with attribute weighted optimization, and then, the optimization is derived. Iterative formula and form show a weighted clustering algorithm to perform cluster analysis on relevant influencing factors in book data management. Digital library is not a library entity: it corresponds to various real social activities of public information management and dissemination and is manifested in various new information resource organizations and information dissemination services. As a fuzzy association rule mining algorithm, FP-growth algorithm is obviously better than Apriori algorithm in execution efficiency. However, due to the lack of fuzzy attributes and the large space complexity, the FP-growth algorithm cannot achieve effective multilayer association rule mining when dealing with large transaction databases, such as library databases. First, the algorithm divides the large-scale book transaction database into several sub-databases according to the transaction of the first item and constructs the corresponding sub-FP-tree structure; then, the parent items that are not frequent items in the hierarchical tree are filtered out in real time to reduce the scanning space.

Fractional salp swarm algorithm: An association rule based privacy-preserving strategy for data sanitization

BackgroundData mining is the process of extracting hidden patterns from huge repositories. Privacy preservation at the time of data shared for mining is a demanding dilemma. Conventional techniques, such as access control and authentication have been used to handle data privacy. Various data sanitization techniques like perturbation, generalization, and sampling are utilized to preserve confidential information from disclosure. MethodThis paper develops a Fractional-Salp swarm algorithm (Fractional-SSA) for data sanitization using privacy preserved data. The Fractional-SSA is developed by integrating Fractional calculus (FC) and the Salp swarm algorithm (SSA). The proposed Fractional-SSA hides sensitive rules considering a large transactional database and recovers the original database against malicious attacks. Here, the random key generation of the sanitization process is performed using the proposed Fractional-SSA algorithm by arbitrarily initializing various keys. In addition, the sanitized database is obtained from sanitization that derives association rules considering certain factors that involve privacy success, information preservation, hiding success, database difference, privacy loss, information loss, hiding failure, and database difference. Finally, the key value gets updated to estimate the best solution. The fitness is obtained using success and failure scenarios. ResultsThe proposed Fractional-SSA offered enhanced efficiency providing the highest privacy success of 0.933, information preservation of 0.487, minimal hiding failure of 0.054, and database difference of 0.008.

Retraction Note to: An improved memory adaptive up-growth to mine high utility itemsets from large transaction databases

Retraction Note to: An improved memory adaptive up-growth to mine high utility itemsets from large transaction databases

Compact Tree Structures for Mining High Utility Itemsets

High Utility Item set Mining (HUIM) from large transaction databases has garnered significant attention as it accounts for the revenue of the items purchased in a transaction. Existing tree-based HUIM algorithms discard unpromising items and require at most two database scans for their construction. Hence, whenever utility threshold is changed, the trees have to be reconstructed from scratch. In this regard, the current study proposes to not only incorporate all the items in the tree structure but compactly represent transaction information. The proposed trees namely- Utility Prime Tree (UPT), Prime Cantor Function Tree (PCFT), and String based Utility Prime Tree (SUPT) store transaction-level information in a node unlike item-based prefix trees. Experiments conducted on both real and synthetic datasets compare the execution time and memory of these tree structures with a proposed Utility Count Tree (UCT) and existing IHUP, UP-Growth trees. Due to transaction-level encoding, these structures consume significantly less memory when compared to the tree structures in the literature.

IEEE Access Special Section Editorial: Utility-Pattern Mining: Theoretical Analytics and Applications

Utility-pattern mining in data has received a lot of attention from the knowledge discovery in database (KDD) community due to its high potential for many applications such as finance, biomedicine, manufacturing, e-commerce, and social media. Current research in utility-pattern mining primarily focuses on discovering patterns of high value (e.g., high profit) in large databases and analyzing/learning important factors (e.g., economic factors) in a data mining process. One of the most popular applications of utility mining is the analysis of large transactional databases to discover high-utility itemsets, which are sets of items that yield a high profit when purchased together.

A new approximate method for mining frequent itemsets from big data

Mining frequent itemsets in transaction databases is an important task in many applications. It becomes more challenging when dealing with a large transaction database because traditional algorithms are not scalable due to the limited main memory. In this paper, we propose a new approach for the approximately mining of frequent itemsets in a big transaction database. Our approach is suitable for mining big transaction databases since it uses the frequent itemsets from a subset of the entire database to approximate the result of the whole data, and can be implemented in a distributed environment. Our algorithm is able to efficiently produce high-accurate results, however it misses some true frequent itemsets. To address this problem and reduce the number of false negative frequent itemsets we introduce an additional parameter to the algorithm to discover most of the frequent itemsets contained in the entire data set. In this article, we show an empirical evaluation of the results of the proposed approach.

The pricing of European non-performing real estate loan portfolios: evidence on stock market evaluation of complex asset sales

Recent empirical evidence raises doubt about the ability of financial market participants to generate information efficient valuations for capital market instruments whose cash flows are related to residual claims and dependent on real estate income. We contribute to this literature with the examination of value implications of non-performing loan (NPL) divestitures in the banking industry during the period 2012–2018. In a first step, we provide descriptive statistics of the European NPL market, which lacks transparency and publicly available basic information on portfolio size and components. We then analyze wealth effects of distressed loan sale announcements for a uniquely large transaction database with 317 NPL deals, which is largely driven by real estate collateral. Our results show positive stock market reactions for vendor banks following NPL divestitures that tend to be driven by real estate collateral and a size effect.

RETRACTED ARTICLE: An improved memory adaptive up-growth to mine high utility itemsets from large transaction databases

High utility itemset (HUI) mining identifies an interesting pattern from transactional databases by taking into consider the utility of each in the transaction for instance, margins or profits. Many candidates are generated for HUIs which reduces the mining performance. Frequent pattern-growth (FP-Growth) algorithm was widely used to discover the frequent itemsets using FP-Tree. But, UP-Tree was constructed to store high utility item set. The mining of UP-Tree by FP-Growth extracts high utility itemset and generates too many candidates. So, UP-Growth and UP-Growth+ was proposed to shorten the candidate itemsets. In UP-Growth, two tactics such as discarding local unpromising items (DLU) and decreasing local node (DLN) were used in FP-Growth. In UP-Growth+, two strategies such as DLU and its estimated node utilities (DNU) and DLN tools for the nodes of internal UP-Tree by evaluated the descendant nodes (DNN) were incorporated in FP-Growth. However, the UP-Growth and UP-Growth+ has the problem of poor spatial and temporal localities. Initially, the UP-Tree is created in available main memory then extended to secondary memory when the transaction is large. The storage of data structure in secondary memory and accessibility is not clearly derived in existing algorithms. In this paper, the spatial locality issues of UP-Growth and UP-Growth+ is solved by rearranging nodes of UP-Tree in a depth first manner and temporal locality issues of UP-Growth and UP-Growth+ is solved by page blocking technique which reorganizes the execution part of UP-Growth and UP-Growth+. The computational part is rearranged. In addition to, the memory management strategy is introduced to minimize the space requirement of high utility itemsets. Thus the proposed Improved Adaptive UP-Growth (IAUP-Growth) and Improved Adaptive UP-Growth+ (IAUP-Growth+) overcomes the spatial and temporal locality problem and effectively reduces the memory usage.

The Pricing of European Non-Performing Real Estate Loan Portfolios: Evidence on Stock Market Evaluation of Complex Asset Sales

Recent empirical evidence raises doubt about the ability of financial market participants to generate information efficient valuations for capital market instruments whose cash flows are related to residual claims and dependent on real estate income. We contribute to this literature with the examination of value implications of non-performing loan (NPL) divestitures in the banking industry during the period 2012 to 2018. In a first step, we provide descriptive statistics of the European NPL market, which lacks transparency and publicly available basic information on portfolio size and components. We then analyze wealth effects of distressed loan sale announcements for a uniquely large transaction database with 317 NPL deals, which is largely driven by real estate collateral. Our results show positive stock market reactions for vendor banks following NPL divestitures that tend to be driven by real estate collateral and a size effect.

Do NPL portfolio sales help reduce banks’ financing costs?

We examine financing cost implications of non-performing loan (NPL) divestitures in the European banking industry. Based on a uniquely large transaction database covering 180 NPL sales, we analyze whether selling banks are able to reduce their financing costs measured by CDS spread changes. We do not find a significant tightening of CDS spreads around NPL divestitures, indicating that European banks are not able to reduce their financing costs by NPL portfolio sales.

Parallel Algorithm for Frequent Itemset Mining on Intel Many-core Systems

Frequent itemset mining leads to the discovery of associations and correlations among items in large transactional databases. Apriori is a classical frequent itemset mining algorithm, which employs iterative passes over database combining with generation of candidate itemsets based on frequent itemsets found at the previous iteration, and pruning of clearly infrequent itemsets. The Dynamic Itemset Counting (DIC) algorithm is a variation of Apriori, which tries to reduce the number of passes made over a transactional database while keeping the number of itemsets counted in a pass relatively low. In this paper, we address the problem of accelerating DIC on the Intel Xeon Phi many-core system for the case when the transactional database fits in main memory. Intel Xeon Phi provides a large number of small compute cores with vector processing units. The paper presents a parallel implementation of DIC based on OpenMP technology and thread-level parallelism. We exploit the bit-based internal layout for transactions and itemsets. This technique reduces the memory space for storing the transactional database, simplifies the support count via logical bitwise operation, and allows for vectorization of such a step. Experimental evaluation on the platforms of the Intel Xeon CPU and the Intel Xeon Phi coprocessor with large synthetic and real databases showed good performance and scalability of the proposed algorithm.

Параллельный поиск частых наборов на многоядерных ускорителях Intel MIC

Association rule mining is one of the basic problems of data mining, which supposes finding strong correlations between itemsets in large transaction database. Association rules are generated from frequent itemsets (itemset is frequent if its items frequent occur together in transactions). The DIC (Dynamic Itemset Counting) algorithm is modification of the classical Apriori algorithm of finding frequent itemsets. DIC tries to reduce the number of passes made over the transaction database while keeping the number of itemsets counted in a pass relatively low. The paper addresses the task of accelerating DIC on the Intel MIC (Many Integrated Core) systems in the case when the transaction database fits into the main memory. The paper presents a parallel implementation of DIC based on OpenMP technology and thread-level parallelism. We exploit the bit-based internal layout for transactions and itemsets. This technique simplifies the support count via logical bitwise operation, and allows for vectorization of such a step. Experiments with large synthetic and real databases showed good performance and scalability of the proposed algorithm.

Paradoxical Correlation Pattern Mining

Given a large transactional database, correlation computing/association analysis aims at efficiently finding strongly correlated items. For traditional association analysis, relationships among variables are usually measured at a global level. In this study, we investigate confounding factors that can help to capture abnormal correlation behaviors at a local level. Indeed, many real-world phenomena are localized to specific markets or subpopulations. Such local relationships may not be visible or may be miscalculated when collectively analyzing the entire data. In particular, confounding effects that change the direction of correlation are a most severe problem because the global correlations alone leads to errant conclusions. To this end, we propose CONFOUND, an efficient algorithm to identify paradoxical correlation patterns (i.e., where controlling for a third item changes the direction of association for strongly correlated pairs) using effective pruning strategies. Moreover, we also provide an enhanced version of this algorithm, called CONFOUND+, which substantially speeds up the confounder search step. Finally, experimental results showed that our proposed CONFOUND and CONFOUND+ algorithms can effectively identify confounders and the computational performance is orders of magnitude faster than benchmark methods.

Computational methods for estimating multinomial, nested, and cross-nested logit models that account for semi-aggregate data

We present a summary of important computational issues and opportunities that arise from the use of semi-aggregate data (where the explanatory data for choice scenarios are not necessarily unique for each decision-maker) in discrete choice models. These data are encountered with large transactional databases that have limited consumer information, a common feature in some transportation planning applications, such as airline itinerary choice modeling. We developed a freeware software package called Larch, written in Python and C++, to take advantage of these kind of data to greatly speed the estimation of discrete choice model parameters. Benchmarking experiments against Stata (a commonly used commercial package), Biogeme (a commonly used freeware package), and ALOGIT (a highly specialized commercial package for discrete choice modeling) based on an industry dataset for airline itinerary choice modeling applications shows that the size of the input estimation files are 50–100 times larger in Stata and Biogeme, respectively. Estimation times are also much faster in ALOGIT and Larch; e.g., for a small itinerary choice problem, a multinomial logit model estimated in ALOGIT or Larch converged in less than one second whereas the same model took almost 15 seconds in Stata and more than three minutes in Biogeme.

Research and Improvement of Apriori Algorithm Based on Hadoop

Abstract Association rules can forcefully get a horizontal relation in the big data, the Apriori algorithm is one of the most significant association rules. Traditional mining based on parallel Apriori algorithms needs much more time in data IO with the increasing size of large transaction database. This paper improves the Apriori algorithm from compressing transactions, reducing the number of scans and simplifying candidate set generation. And then the improved algorithm is parallelized on the Hadoop framework. The experiments show that this improved algorithm is suitable for large-scale data mining and has good scalability and effectiveness.

Construction of real estate price indices for developing housing markets

PurposeThe purpose of this paper is to test whether temporal aggregation matters when constructing hedonic house price indices for developing markets using Ghana as a case study.Design/methodology/approachMonthly, quarterly, semi-yearly and yearly hedonic price indices are constructed and six null hypotheses are tested using the F-ratios to examine the temporal aggregation effect.FindingsThe results show that temporal aggregation may not be a serious issue when constructing hedonic house price indices for developing markets as a result of the smaller sample size which these markets normally have. At even 10 per cent significance level, none of the F-ratios estimated is statistically significant. Analysis of the mean returns and volatilities reveal that indices constructed at the lower level of temporal aggregation are very volatile, suggesting that the volume of transactions can affect the level of temporal aggregation, and so, the temporal aggregation level should not be generalised, as is currently observed in the literature.Originality/valueThe diversification importance of real estate and the introduction of real estate derivatives and home equity insurance as financial products call for the construction of robust and accurate real estate indices in all markets. While almost all empirical research recommends real estate price indices to be conducted at the lower level of temporal aggregation, these studies are largely conducted in developed markets where transactions take place frequently and large transaction databases exist. Unfortunately, little is known about the importance of temporal aggregation effect when constructing indices for developing real estate markets. This paper contributes to fill these gaps.

FHN: An efficient algorithm for mining high-utility itemsets with negative unit profits

High utility itemset mining is an emerging data mining task, which consists of discovering highly profitable itemsets (called high utility itemsets) in very large transactional databases. Many algorithms have been proposed to efficiently discover high utility itemsets but most of them assume that items may only have positive unit profits. However, in real-world transactional databases, items (products) often have positive or negative unit profits. Mining high utility itemsets in a transactional database where items have positive or negative unit profits is a computationally expensive task, and it is thus desirable to design more efficient algorithms. To address this issue, we propose an efficient algorithm named FHN (Faster High-Utility itemset miner with Negative unit profits). It relies on a novel PNU-list structure (Positive-and-Negative Utility-list) structure to efficiently mine high utility itemsets, while considering both positive and negative unit profits. Moreover, several pruning strategies are introduced in FHN to reduce the number of candidate itemsets, and thus enhance the performance of FHN. Extensive experimental results on both real-life and synthetic datasets show that the proposed FHN algorithm is in general two to three orders of magnitude faster and can use up to 200 times less memory than the state-of-the-art algorithm HUINIV-Mine. Moreover, it is shown that FHN performs especially well on dense datasets.

Reducing thread divergence in GPU‐based bees swarm optimization applied to association rule mining

SummaryThe association rules mining (ARM) problem is one of the most important problems in the area of data mining. It aims at finding all relevant association rules from transactional databases. It is CPU time intensive and requires a huge computing power when dealing with large transactional databases. To deal with this issue, Graphics Processing Units (GPUs) are a powerful tool to speed up the search process. However, their performance is closely subject to thread/branch divergence resulting from the single instruction multiple data parallel model of GPUs. In this paper, we propose three approaches based on database reorganization, aiming to reduce thread divergence in GPU‐based bees swarm optimization metaheuristic for ARM, respectively, named block‐based reordering, transactions‐based reordering, and transactions‐based reordering with median value. Theoretical and experimental studies have been carried out using well‐known large ARM instances. The experiments have been performed on an Intel Xeon 64 bit quad‐core processor E5520 coupled to Nvidia Tesla C2075 448 cores. The results show that the proposed approaches minimize considerably the number of thread divergence and improve the overall execution time. Indeed, the number of thread divergence occurrences has been reduced by up to eight times making the execution much faster. Copyright © 2016 John Wiley & Sons, Ltd.

Frequent Itemset Mining in Data Mining: A Survey

Data mining is process of extracting useful information from different perspectives. Frequent Item set mining is widely used in financial, retail and telecommunication industry. The major concern of these industries is faster processing of a very large amount of data. Frequent item sets are those items which are frequently occurred. So we can use different types of algorithms for this purpose. Frequent Item se performed Apriori, FP-tree, Eclat , and RARM the work in this paper, we have analyzed algorithms for finding frequent patterns with the purpose of discovering how these algorithms can be used to obtain frequent patterns over large transactional databases. This has been presented in the form of a compa rative study of the following algorithms: Apriori, Freque nt Pattern (FP) Growth Rapid Association Rule Mining (RARM) and ECLAT algorithm frequent pattern mining algorithms. This study also focuses on each of the algorithm's advantages, disadvantages and limitations for finding patterns among large item sets in database systems. Keywordsrule, Frequent Item, Data Mining