Efficient Mining Algorithm Research Articles

Discovering and Ranking Urban Social Clusters Out of Streaming Social Media Datasets

ABSTRACTUrban social media mining is the process of discovering urban patterns from spatio‐temporal social media datasets. Urban social clusters are the clusters formed by the social media posts of users living in cities at a certain time and place. Discovering and identifying urban social clusters is of great importance for urban and regional planning, target audience identification, a better understanding of city dynamics and so forth. Discovering and ranking urban social clusters out of streaming social media datasets require efficient filtering approaches and mining algorithms. In the literature, there are several studies performed that address the discovery of the importance of urban clusters. Most of these studies take into account the spatial expansions over time and the changes in the numbers of elements within clusters when identifying the significance of urban clusters. However, in contrast to these studies, we have also considered cluster temporal formation stability, spatial density variation, and the impact of meta‐information on urban social clusters. In this study, Temporal, Spatial, and Meta Important Urban Social Clusters Miner (TSMIUSC‐Miner) algorithm is proposed. In the proposed algorithm, urban social clusters are discovered, and their importance relative to each other are compared and ranked. The temporal, spatial and meta importance scores of the clusters are calculated and then, the clusters that satisfy predefined score thresholds are discovered. The performance of the proposed TSMIUSC‐Miner algorithm compared with that of a naive approach using real‐life streaming Twitter/X dataset. The results showed that the proposed TSMIUSC‐Miner algorithm outperforms the naive approach in terms of execution time.

A related degree-based frequent pattern mining algorithm for railway fault data

It is of great significance to improve the efficiency of railway production and operation by realizing the fault knowledge association through the efficient data mining algorithm. However, high utility quantitative frequent pattern mining algorithms in the field of data mining still suffer from the problems of low time-memory performance and are not easy to scale up. In the context of such needs, we propose a related degree-based frequent pattern mining algorithm, named Related High Utility Quantitative Item set Mining (RHUQI-Miner), to enable the effective mining of railway fault data. The algorithm constructs the item-related degree structure of fault data and gives a pruning optimization strategy to find frequent patterns with higher related degrees, reducing redundancy and invalid frequent patterns. Subsequently, it uses the fixed pattern length strategy to modify the utility information of the item in the mining process so that the algorithm can control the length of the output frequent pattern according to the actual data situation and further improve the performance and practicability of the algorithm. The experimental results on the real fault dataset show that RHUQI-Miner can effectively reduce the time and memory consumption in the mining process, thus providing data support for differentiated and precise maintenance strategies.

A Novel and Efficient Spatio-Temporal Co-location Pattern Mining Algorithm

Colocation pattern mining approaches aim at discovering neighboring relationships of distinct spatial features in geographic and temporal space.With big spatio-temporal dataset, there is large number of patterns often discovered. Then it is of importance to discover meaningful and patterns of interest which come as an aid in applications in use for humans and commercial use. To discover interesting patterns, we present in this article a co-location pattern mining algorithmic framework by considering neighbourhood, clustering, hashing, and mining methods. Neighbourhood relationship describes the closeness between the entities. The results of neighbourhood could be varied by varying the distance threshold. These objects exhibiting neighbouring entities are grouped using clustering technique. Clustering is a classical research approach that produces grouping of the entities. A clustering technique has been presented in the paper for spatio-temporal dimension that offers the advantage of faster grouping based on the neighbourhood relationship. Finally, a hash structure is utilized for faster access and retrieval of patterns. The proposed mining algorithm along with the distance and time threshold efficiently discovers the interesting spatio-temporal patterns and validates the patterns. The experiment conducted shows that the proposed co-location algorithm method yields effective and efficient outcome.

X-FSPMiner: A Novel Algorithm for Frequent Similar Pattern Mining

Frequent similar pattern mining (FSP mining) allows for finding frequent patterns hidden from the classical approach. However, the use of similarity functions implies more computational effort, necessitating the development of more efficient algorithms for FSP mining. This work aims to improve the efficiency of mining all FSPs when using Boolean and non-increasing monotonic similarity functions. A data structure to condense an object description collection, named FV-Tree , and an algorithm for mining all FSPs from the FV-Tree , named X-FSPMiner , are proposed. The experimental results reveal that the novel algorithm X-FSPMiner vastly outperforms the state-of-the-art algorithms for mining all FSPs using Boolean and non-increasing monotonic similarity functions.

Research on Physical Training Helping College Students’ Physical Fitness and Health Quality Development in the Context of “Healthy China”

Abstract In the face of the current dilemma and predicament of students’ physical health, it has become inevitable to incorporate health education into the national education system, with the help of physical training to reshape cognition, establish beliefs, guide behavior, and other effective means, so as to achieve the fundamental purpose of improving the physical health of students. This paper takes college students’ physical health as the research object, carries out the research on efficient association rule mining algorithm for BMI index, uses Apriori association rule algorithm in R language data mining to carry out in-depth mining of physical health test data, and carries out in-depth comparative analysis and exploration of intrinsic information of physical test data of two academic years of 2022 grade in a school in a more detailed and in-depth way. A scientific guidance program for physical training is created using the collaborative filtering algorithm to support the high-quality development of college students’ physical fitness and health. The results of the study found that the correlation coefficient between physical training behavior and BMI index r = -0.753, confidence test value P = 0.0004 < 0.05, indicating that there is a significant negative correlation between students’ physical training behavior and BMI. That is, physical education and sports can significantly promote the physical health of college students, and ultimately, their BMI should fall within the normal range.

Notice of Retraction: CHN: an efficient algorithm for mining closed high utility itemsets with negative utility

Retracted.

DRIVE: Dockerfile Rule Mining and Violation Detection

A Dockerfile defines a set of instructions to build Docker images, which can then be instantiated to support containerized applications. Recent studies have revealed a considerable amount of quality issues with Dockerfiles. In this article, we propose a novel approach, Dockerfiles Rule mIning and Violation dEtection ( DRIVE ), to mine implicit rules and detect potential violations of such rules in Dockerfiles. DRIVE first parses Dockerfiles and transforms them to an intermediate representation. It then leverages an efficient sequential pattern mining algorithm to extract potential patterns. With heuristic-based reduction and moderate human intervention, potential rules are identified, which can then be utilized to detect potential violations of Dockerfiles. DRIVE identifies 34 semantic rules and 19 syntactic rules including 9 new semantic rules that have not been reported elsewhere. Extensive experiments on real-world Dockerfiles demonstrate the efficacy of our approach.

Abstract A164: Activity of dual BET and CDK9 inhibition in Pancreatic Ductal Adenocarcinoma informed from KinderMiner Prediction

Abstract Background: Pancreatic ductal adenocarcinoma (PDAC) is the third leading cause of cancer mortality in the United States (US) with a 10% 5-year survival. Acquired resistance plays a critical role in progression, however targeted therapeutics have shown minimal clinical activity. There remains an unmet need to provide combination targeted therapies across the breadth of clinical development. KinderMiner is a highly efficient text mining algorithm developed to predict association with search terms across publicly available research. BMS986158 is a novel bromodomain and extraterminal (BET) inhibitor in early phase clinical trials, recently reported to have preclinical activity in combination with chemotherapy. Methods: Potential genes for targeting in combination with BMS986158 were queried using KinderMiner (KM) and Serial KinderMiner (SKiM). Search terms included synonymous terms of “drug resistance” and “BET”. BMS986158 was assayed for single agent activity at 72h and 144h. Lead agents in active clinical trials were screened for synergistic activity. All assays were performed with patient derived organoids (PCOs) in a low volume format with high content imaging, using dose titration in combination grid to assay synergy. Viability was assessed using 3D CellTiterGlo (CTG, Promega Inc) (50% v/v) with synergy scores determined by SynergyFinder 3.0 for Bliss Independence (Bliss), Highest Single Agent (HSA), and Loewe Additivity (Loewe). Results: A KM/SKiM query was performed against >30M primary literature reports resulting in 39 unique genes with significant relationship with BET. CDK9 was selected as a lead gene target based on CDK9 inhibitor, AZD4573, in active clinical trials with highly significant prediction from text mining screen (p<0.001). BMS986158 was found to have nanomolar single agent IC50 activity across multiple primary PDAC PCOs (n=5) that improved from 72h [43-411nM] in comparison to 144h [12-54nM]. AZD4573 was found to have single agent IC50 ranging from 18-246nM across 3 primary cultures. Therapeutically relevant dosing of 6.25nM BMS986158 and 12.5nM AZD4573 in combination was found to induce significant response including normalized change in diameter relative to control, single agent BMS986158, and AZD4573 (effect sizes=1.7, 1.4, 0.7) The combination of BMS986158 and AZD4573 was also found to have augmented activity with positive synergy scores across all reference models: Bliss=3.47, HSA=8.81, and Loewe=15.92. Conclusion: Here, we adapt a novel text-mining algorithm, KinderMiner, to screen agents for combination with BMS986158 to inform combination therapeutics. The combination of BMS986158 and AZD4573 was shown to have improved activity in a model of PDAC PCOs. Validation of the combination efficacy across different PDAC PCOs is ongoing to compare this activity across a diversity of mutational profiles. Future directions will assay the mechanistic modeling to characterize the interaction between transcriptional regulation and the induction of apoptosis. Citation Format: Austin Stram, Luke Koeppel, Rob Millikin, Eleanor Riedl, Md Shahadat Hossan, Ethan S Lin, Ron Stewart, Jeremy D Kratz. Activity of dual BET and CDK9 inhibition in Pancreatic Ductal Adenocarcinoma informed from KinderMiner Prediction [abstract]. In: Proceedings of the AACR-NCI-EORTC Virtual International Conference on Molecular Targets and Cancer Therapeutics; 2023 Oct 11-15; Boston, MA. Philadelphia (PA): AACR; Mol Cancer Ther 2023;22(12 Suppl):Abstract nr A164.

FAC-fed: Federated adaptation for fairness and concept drift aware stream classification

Federated learning is an emerging collaborative learning paradigm of Machine learning involving distributed and heterogeneous clients. Enormous collections of continuously arriving heterogeneous data residing on distributed clients require federated adaptation of efficient mining algorithms to enable fair and high-quality predictions with privacy guarantees and minimal response delay. In this context, we propose a federated adaptation that mitigates discrimination embedded in the streaming data while handling concept drifts (FAC-Fed). We present a novel adaptive data augmentation method that mitigates client-side discrimination embedded in the data during optimization, resulting in an optimized and fair centralized server. Extensive experiments on a set of publicly available streaming and static datasets confirm the effectiveness of the proposed method. To the best of our knowledge, this work is the first attempt towards fairness-aware federated adaptation for stream classification, therefore, to prove the superiority of our proposed method over state-of-the-art, we compare the centralized version of our proposed method with three centralized stream classification baseline models (FABBOO, FAHT, CSMOTE). The experimental results show that our method outperforms the current methods in terms of both discrimination mitigation and predictive performance.

Discovering High Utility Episodes in Sequences

Sequence data is more commonly seen than other types of data (e.g., transaction data) in real-world applications. For the mining task from sequence data, several problems have been formulated, such as sequential pattern mining, episode mining, and sequential rule mining. As one of the fundamental problems, episode mining has often been studied. The common wisdom is that discovering frequent episodes is not useful enough. In this paper, we propose an efficient utility mining approach, namely UMEpi: <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</u> tility <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> ining of high-utility <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Epi</u> sodes from complex event sequences. We propose the concept of remaining utility of episodes and achieve a tighter upper bound, namely episode-weighted utilization ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> ), which will provide better pruning. Thus, the optimized <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> -based pruning strategies can achieve better improvements in mining efficiency. The search space of UMEpi w.r.t. a prefix-based lexicographic sequence tree is spanned and determined recursively for mining high-utility episodes, by prefix-spanning in a depth-first way. Finally, extensive experiments on four real-life datasets demonstrate that UMEpi can discover the complete high-utility episodes from complex event sequences. Furthermore, the improved variants of UMEpi significantly outperform the baseline in terms of execution time, memory consumption, and scalability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement</i> —This article contributes to the problem of utility-based episode discovery, which is a challenging task in artificial intelligence and data science. To the best of our knowledge, it is the first article that introduces the concept of the remaining utility of an episode in an event sequence and then formulates an alternative definition of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> , which is able to provide an accurate formulation of the upper bound. The designed UMEpi algorithm can be a benchmark for utility-based episode discovery. UMEpi addresses several challenges, and the optimized <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> -based pruning strategies can achieve better improvements in mining efficiency on massive datasets. It is well known that episode discovery has been successfully applied in a wide range of real-world applications and various domains. In summary, this correct and efficient utility episode mining algorithm contributes to the artificial intelligence systems in many applications, especially complex event processing and analysis.

A Parallel High-Utility Itemset Mining Algorithm Based on Hadoop

High-utility itemset mining (HUIM) can consider not only the profit factor but also the profitable factor, which is an essential task in data mining. However, most HUIM algorithms are mainly developed on a single machine, which is inefficient for big data since limited memory and processing capacities are available. A parallel efficient high-utility itemset mining (P-EFIM) algorithm is proposed based on the Hadoop platform to solve this problem in this paper. In P-EFIM, the transaction-weighted utilization values are calculated and ordered for the itemsets with the MapReduce framework. Then the ordered itemsets are renumbered, and the low-utility itemsets are pruned to improve the dataset utility. In the Map phase, the P-EFIM algorithm divides the task into multiple independent subtasks. It uses the proposed S-style distribution strategy to distribute the subtasks evenly across all nodes to ensure load-balancing. Furthermore, the P-EFIM uses the EFIM algorithm to mine each subtask dataset to enhance the performance in the Reduce phase. Experiments are performed on eight datasets, and the results show that the runtime performance of P-EFIM is significantly higher than that of the PHUI-Growth, which is also HUIM algorithm based on the Hadoop framework.

Optimized hadoop map reduce system for strong analytics of cloud big product data on amazon web service

Because of the rapid increase of data in the cloud of Amazon Web Service (AWS), the traditional methods for analyzing this data are not good and inappropriate, so unconventional methods of analysis have been proposed by many data scientists such as concurrent/ parallel techniques to meeting the requirements of performance and scalability entailed in such big data analyses. In this paper we are used Hadoop Map Reduce system that contains Hadoop Distributed File System (HDFS) and Hadoop cluster. We optimized it by combining it with five efficient Data Mining (DM) algorithms such as Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), Correlative Naïve Bayes classifier (CNB), and Fuzzy CNB (FCNB) for strong analytics of cloud big data. The proposed system applied on product review data that taken form the cloud of AWS. The Evaluation of Hadoop Map Reduce done with important benchmarks as Mean Absolute Percentage Error (MPAE), Root Mean Square Error (RMSE), and runtime for word count, sort, inverted index. Also, the evaluation of DM models with Hadoop Map Reduce system done by using accuracy, sensitivity, specificity, memory, and running time. Experiments have shown that FCNB is effective in addressing the problem of big data.

GPU-Based Efficient Parallel Heuristic Algorithm for High-Utility Itemset Mining in Large Transaction Datasets

GPU-Based Efficient Parallel Heuristic Algorithm for High-Utility Itemset Mining in Large Transaction Datasets

Exploring influential nodes using global and local information

In complex networks, key nodes are important factors that directly affect network structure and functions. Therefore, accurate mining and identification of key nodes are crucial to achieving better control and a higher utilization rate of complex networks. To address this problem, this paper proposes an accurate and efficient algorithm for critical node mining. The influential nodes are determined using both global and local information (GLI) to solve the shortcoming of the existing key node identification methods that consider either local or global information. The proposed method considers two main factors, global and local influences. The global influence is determined using the K-shell hierarchical information of a node, and local influence is obtained considering the number of edges connected by the node and the given values of adjacent nodes. The given values of adjacent nodes are determined based on the degree and K-shell hierarchical information. Further, the similarity coefficient of neighbors is considered, which enhances the differentiation degree of the adjacent given values. The proposed method solves the problems of the high complexity of global information-based algorithms and the low accuracy of local information-based algorithms. The proposed method is verified by simulation experiments using the SIR and SI models as a reference, and twelve typical real-world networks are used for the comparison. The proposed GLI algorithm is compared with several common algorithms at different periods. The comparison results show that the GLI algorithm can effectively explore influential nodes in complex networks.

Optimization and verification of single pilot operations model for commercial aircraft based on biclustering method

Commercial aircraft crews have experienced a trend from five-person crew to dual-pilot crew. Arised from both technological and market demands, Single Pilot Operations (SPO) is considered an important development direction in modern aviation technology. In this paper, starting from Dual-Pilot Operations (DPO), the piloting process, decision-making process and decision-making mode of DPO for commercial aircraft are studied to obtain the operational requirements of SPO. Then, based on above analysis, the operational mechanism of SPO is studied and the core technology of SPO mode is proposed. Next, a new closed frequent bicluster mining algorithm named FsCluster is proposed for the optimization of the SPO model, and the other efficient bicluster mining algorithm named TsCluster is proposed for the analysis and verification of the SPO model. Finally, a typical flight phase scenario is modelled by Magic System of System, and combined with the proposed algorithms for analysis and verification to determine whether the SPO mode can meet the DPO requirements.

An efficient algorithm for mining closed high utility itemsets over data streams with one dataset scan

An efficient algorithm for mining closed high utility itemsets over data streams with one dataset scan

Efficient algorithms for mining closed and maximal high utility itemsets

Closed high utility itemsets (CHUIs) and maximal high utility itemsets (MaxHUIs) are two important concise representations of HUIs. Discovering these itemsets is important because they are lossless and compact, i.e., they provide a concise summary of all HUIs that can be orders of magnitude smaller. In addition, it can be more efficient to extract these representations than it would be to extract all HUIs. Mining the concise representations of HUIs is also an important step toward the generation of nonredundant high utility association rules that can reveal meaningful information to decision-makers. However, although several algorithms have been designed to mine these representations, such as EFIM-Closed, HMiner-Closed, and CHUI-Miner(Max), they have long runtimes, high memory usage, and scalability issues, especially for dense and large datasets. To address this issue, this paper proposes two efficient algorithms named C-HUIM and MaxC-HUIM for mining CHUIs, and simultaneously mining both CHUIs and MaxHUIs, respectively. These algorithms use a novel weak upper bound on the utility, which is strictly tighter than traditional upper bounds, and a corresponding pruning strategy called SPWUB to quickly eliminate low utility itemsets. The algorithms also include two novel search space reduction strategies named PSNonCHUB and LPSNonCHUB. The PSNonCHUB strategy only requires checking the inclusion relationship among a small number of itemsets, while LPSNonCHUB does not perform any inclusion check. In addition, the algorithms adopt a structure named MPUN-list to efficiently store and calculate information about each itemset’s utility and support. Experimental results show that the proposed algorithms can be more than 100 times faster, are more memory efficient, and have better scalability than the state-of-the-art algorithms.

Web Log Analysis and Security Assessment Method Based on Data Mining.

Web content mining describes the classification, clustering, and attribute analysis of a large number of text documents and multimedia files on the web. Special tasks include retrieval of data from the Internet search engine tool W; structured processing and analysis of web data. Today's blog analysis has security concerns. We do experiments to investigate its safety. Through experiments, we draw the following conclusions: (1) Web log extraction can use efficient data mining algorithms to systematically extract logs from web servers, then determine the main access types or interests of users, and then to a certain extent, based on the discovered user patterns, analyze the user's access settings and behavior. (2) No matter in the test set or the mixed test set, the curve value of deep mining is very stable, the curve value has been kept at 0.95, and the curve value of fuzzy statistics method and quantitative statistics method is stable within the interval of 0.90-095. The results also show that the data mining method has the highest identification accuracy and the best security performance. (3) Web usage analysis requires data abstraction for pattern discovery. This data abstraction can be achieved through data preprocessing, which introduces different formats of web server log files and how web server log data is preprocessed for web usage analysis. One of the most critical parts of the web mining field is web log mining. Web log mining can use powerful data mining algorithms to systematically mine the logs in the web server and then learn the user's access or preferred interests and then conduct a certain degree of user preferences and behavior patterns according to the discovered user patterns. Based on the above analysis, the current web log analysis is faced with security problems. We conduct experiments to study to verify the security performance of web logs and draw conclusions through experiments.

Efficient high-utility occupancy itemset mining algorithm on massive data

Mining interesting itemsets on massive data is a necessary topic in data mining. Nowadays, most studies use frequency or utility as primary measure. However, using these two measures individually has its own limitations. For example, itemsets with high frequencies may have low profits while itemsets with high utilities perhaps appear occasionally, so they might be misleading. In addition, the existing algorithms can only deal with small-medium scale database, and their performances degrade significantly when data is expanded. To address these drawbacks, this paper proposes a novel high utility occupancy itemset mining algorithm SHO (Suffix-based High-utility Occupancy itemset mining), it considers both quantities and profits of itemsets. SHO designs the algorithm from suffix-based partitioning, generation pruning and itemsets linking, it can mine high utility occupancy itemsets on large-scale database effectively. At the beginning, the database are divided into some non-overlapping suffix-based partitions and stored in vertical format, then the support and utility occupancy of itemset can be calculated in a certain partition instead of traversing total database. Besides, two optimization strategies and four pruning strategies are proposed to make SHO faster. The extensive experiments show that SHO is much better than the current state-of-the-art algorithm, the efficiency can be improved up to 3 orders of magnitude.

Efficient Uncertain Sequence Pattern Mining Based on Hadoop Platform

In the Internet of Things (IoT) era, information is collected by sensor devices, resulting in data loss or uncertain data and other consequences. We need to represent the uncertain data collected using probabilities to extract the useful information for production and application from a huge indeterminate data warehouse. The data in the database has a particular order in time or space, so the High-Utility Probability Sequential Pattern Mining (HUPSPM) has become a new investigation and analysis topic in data processing. After the progress of timestamp, many efficient algorithms for sequential mining have been developed. However, these algorithms have a limitation: they can only be executed in a stand-alone environment and are only suitable for small datasets. Therefore, introducing an advanced graph framework for processing large datasets addresses the shortcomings of the existing methods. The proposed algorithm can avoid repeated database searching, splitting the database, and improve the parallel computing capability. The initial database is pruned according to the existing pruning strategy to effectively reduce the number of candidate sets effectively. Experiments show that the algorithm presented in this paper has excellent advantages in mining high-utility probability sequences in large datasets.