Data Mining Environment Research Articles

The Role of Artificial Intelligence in Managing Knowledge in a Data Mining Environment through Knowledge Reusability

Current research attempts to demonstrate the prospects of Artificial Intelligence (AI) for knowledge management and the resulting systems in a three-dimensional environment. We analytically examine underlying characteristics of knowledge management, such as knowledge sharing, application, and storage, in the context of tacit and explicit knowledge in a data mining environment. The crucial role of tact and explicit knowledge and its reusability, especially in a data mining environment, has been considered in present studies. It is believed that there is a need to develop a professional model incorporating Artificial Intelligence (AI) to benefit from knowledge management in companies, especially about tacit and explicit knowledge, and its reusability. The likely role of AI applications during various knowledge management processes and the benefits of AI for knowledge management systems that leverage tacit and explicit knowledge and knowledge reusability were highlighted

Blockchain-based Decentralized Metaverse Systems, Industrial Artificial Intelligence of Things, and Spatial Data Mining and Acoustic Environment Recognition Algorithms in Realistic 3D Simulation Environments

Blockchain-based Decentralized Metaverse Systems, Industrial Artificial Intelligence of Things, and Spatial Data Mining and Acoustic Environment Recognition Algorithms in Realistic 3D Simulation Environments

Novel method for optimizing performance in resource constrained distributed data streams

The Big Data Era has presented many opportunities for using data mining techniques to discover knowledge patterns across large and diverse collections of data where the volume of data is growing at an exponential rate. Recent approaches to Distributed Data Mining (DDM) have focused on addressing the heterogeneous nature of data sources. However, such approaches do not prioritize the reduction of data communication costs which could be prohibitive in large scale sensor networks where bandwidth is a limited resource. In fact, higher communication and computational costs are the two most prominent problems that have been encountered in heterogeneous distributed environments. Moreover, an effort to decrease the communications load in the distributed environment has an adverse influence on the classification accuracy. Therefore, the research challenge lies in maintaining a balance between transmission cost, computational cost, and accuracy. This paper proposes an algorithm Performance Optimizer in Distributed Stream Mining (PODSM) based on Bayesian Inference to reduce the communication volume and resource time in a heterogeneous distributed data mining environment while retaining prediction accuracy. The approach used in this work exploits the past data for calculating statistics and these statistics are then utilized for the new data. In other words, it imparts the ability to learn from experiences. As a result, our experimental evaluation reveals that a significant reduction in the communication load and an improvement in classification response time can be achieved across a diverse range of dataset types. Reduction of 34.66% was obtained with regard to communication overhead for one of the datasets with huge savings of nearly 27% in resource time. Importantly, instead of showing a negative effect on accuracy, this dataset observes an increment of 0.44% in accuracy.

Enhancing replica management in a cloud environment using data mining based dynamic replication algorithm

Cloud computing has recently gained a lot of popularity and attention from the research community. One of the many on-demand services that large-scale applications provide to cloud customers is storage, which accumulates more generated data and subsequently leads to the need for storage. Despite the fact that users can use the cloud to store and provide the type of storage that desire, it still takes a significant amount of time to store and retrieve data due to the large accumulation of data. Due to the need to improve data availability, response time, reliability, and migration costs, the current storage engine needs to be replicated across multiple sites. When copies are properly distributed, data replication speeds up execution. The biggest challenges in data replication are choosing which data to replicate, where to put it, how to manage replication, and how many replicas it needs. Therefore, various studies have been carried out on some data mining-based data replication systems to evaluate replication issues and manage cloud storage. In most cases, data replication in a data mining environment is done using data mining along with a replication algorithm and a data grid policy. In addition, this paper addresses replica management issues and proposes affordable data replication in the cloud that satisfies all Quality of Service (QoS) requirements.

Transcriptional milestones in Dictyostelium development.

Dictyostelium development begins with single-cell starvation and ends with multicellular fruiting bodies. Developmental morphogenesis is accompanied by sweeping transcriptional changes, encompassing nearly half of the 13,000 genes in the genome. We performed time-series RNA-sequencing analyses of the wild type and 20 mutants to explore the relationships between transcription and morphogenesis. These strains show developmental arrest at different stages, accelerated development, or atypical morphologies. Considering eight major morphological transitions, we identified 1371 milestone genes whose expression changes sharply between consecutive transitions. We also identified 1099 genes as members of 21 regulons, which are groups of genes that remain coordinately regulated despite the genetic, temporal, and developmental perturbations. The gene annotations in these groups validate known transitions and reveal new developmental events. For example, DNA replication genes are tightly coregulated with cell division genes, so they are expressed in mid-development although chromosomal DNA is not replicated. Our data set includes 486 transcriptional profiles that can help identify new relationships between transcription and development and improve gene annotations. We show its utility by showing that cycles of aggregation and disaggregation in allorecognition-defective mutants involve dedifferentiation. We also show sensitivity to genetic and developmental conditions in two commonly used actin genes, act6 and act15, and robustness of the coaA gene. Finally, we propose that gpdA is a better mRNA quantitation standard because it is less sensitive to external conditions than commonly used standards. The data set is available for democratized exploration through the web application dictyExpress and the data mining environment Orange.

Developing machine learning models for relative humidity prediction in air-based energy systems and environmental management applications

The prediction of relative humidity is a challenging task because of its nonlinear nature. The machine learning-based prediction strategies have attained significant attention in tackling a broad class of challenging nonlinear and complex problems. The random forest algorithm is a well-proven machine learning algorithm due to its ease of training and implementation, as it requires minimal preprocessing. The random forest algorithm has hitherto not been employed for estimating air quality parameters, such as relative humidity. In this study, the random forest approach is implemented to estimate the relative humidity as a function of dry- and wet-bulb temperatures. A well-known commercial process simulator called Aspen HYSYS® V10 is linked with MATLAB® version 2019a to establish a data mining environment. The robustness of the prediction model is evaluated against varying wet-bulb depressions. There is high absolute deviation that indicates a lower prediction performance of the model against the higher wet-bulb depression i.e., ~20.0 °C. The random forest model can predict relative humidity with a 1.1% mean absolute deviation compared to the values obtained through Aspen HYSYS. The performance of the RF estimation model is also compared with a well-known support vector regression model. The random forest model demonstrates 74.4% better performance than the support vector machine model for the problem of interest, i.e., relative humidity estimation. This study will significantly help the practitioners in efficient designing of air-dependent energy systems as well as in better environmental management through rigorous prediction of relative humidity.

A Tool for Automatic Generation of Data Mining Practice Environments

Data analysis subjects are an important part of the computer science degree. In these subjects, seminars and practical sessions where students can practice with different data mining algorithms and techniques are a paramount part. This usually involves the installation and deployment of a common development environment for all the students including libraries, databases, datasets and so forth. Due to the heterogeneity of these environments and the characteristics of the undergraduates, such a configuration and deployment might be an important task in the seminars schedule. Consequently, the present work puts forward an open-source tool for the automatic deployment of data-mining environments based on Docker containers. By means of this tool, the teacher can specify the parameters of the environment to deploy. This results in a set of virtual images comprising the whole practice environment that can be easily distributed among students. This way, we meaningfully optimize the aforementioned deployment step and we also guarantee that all students work with the same practice environment. Finally, we have evaluated the performance of the tool and the level of satisfaction of the students with the generated environments showing quite promising results.

Data Hiding

With the widespread growth of digital information and improved internet technologies, the demand for improved information security techniques has significantly increased due to privacy leakage, identity theft, illegal copying, and data distribution. Because of this, data hiding approaches have received much attention in several application areas. However, those approaches are unable to solve many issues that are necessary to measure in future investigations. This survey provides a comprehensive survey on data hiding techniques and their new trends for solving new challenges in real-world applications. The notable applications are telemedicine, 3D objects, mobile devices, cloud/distributed computing and data mining environments, chip and hardware protection, cyber physical systems, internet traffic, fusion of watermarking and encryption, joint compression and watermarking, biometric watermarking, watermarking at the physical layer, and many other perspectives. Further, the potential issues that existing approaches of data hiding face are identified. I believe that this survey will provide a valuable source of information for finding research directions for fledgling researchers and developers.

A hybrid-security model for privacy-enhanced distributed data mining

This study encompasses the proposal of a novel hybrid-security model that incorporates the benefits of both the centralised data mining system as well the distributed data mining system. The hybrid model provides two levels of security. The first security level perturbs the individual datasets by transforming it into a non-understandable form using the four-dimensional rotation transformation, and the second security level helps in performing secure distributed data mining using the ratio of secure summation protocol. The trivial data mining techniques, such as k-means clustering technique and naïve Bayes classification technique, verifies the efficiency and accuracy of the hybrid security model. The hybrid security model provides security to sensitive data without compromising the quality of the data. The accuracy obtained in classification task and clustering task using naïve Bayes and k-means technique is high when different datasets in a privacy-enhanced distributed data mining environment verify the working of the hybrid security model.

R as an environment for data mining of process mineralogy data: A case study of an industrial rougher flotation bank

Through a series of in-house routines of R, an open-source programming language for statistical computing, statistical analysis is applied to automated process mineralogy data to describe the performance of an industrial scheelite rougher flotation bank. These routines allow (1) freeing the user from the limitations of the menu-driven built-in processing and spreadsheet-based analyses routines; in particular when processing data from several streams, and (2) a more flexible manipulation of the data at any level of aggregation.In an illustration case study, it was determined that ideally floating scheelite particles are coarser than 40 µm and are more than 40% liberated. Most of the scheelite lost to the rougher tailings stream is either ultrafine or coarse with little surface liberation and associated with silicates. More importantly, the presence of a depressant does not permit the selective flotation of scheelite from other semi-soluble salt-type minerals such as calcite. This is linked to particle size, as there appears to be some overgrinding before the rougher flotation. While the impact of the depressant requires more observation, less fine grinding could already potentially improve the concentrate grade and decrease operational costs.

Managing privacy of sensitive attributes using fuzzy-based data transformation methods in privacy preserving data mining environment

When we extract personal, sensitive and business information in data mining applications, then certain problems occurs. Privacy attack occurs due to the misuse of individual information. In central...

Managing privacy of sensitive attributes using fuzzy-based data transformation methods in privacy preserving data mining environment

When we extract personal, sensitive and business information in data mining applications, then certain problems occurs. Privacy attack occurs due to the misuse of individual information. In centralised database environment, data transformation methods in fuzzy-based data in the field of privacy preserving clustering are proposed in this paper. In first case, a fuzzy data transformation method is proposed and different experiments are conducted by changing the fuzzy membership functions such as Z-shaped fuzzy membership function, Triangular fuzzy membership function, Gaussian fuzzy membership function to transform the original dataset. In second case, a hybrid method is proposed as a combination of fuzzy data transformation approach which is specified in first case and random rotation perturbation (RRP). The experimental outcome verified that the hybrid approach permits finest results for every member functions.

Visualization in a Data Mining Environment from a Human Computer Interaction Perspective

With the aim of providing support to the design, analysis and evaluation of result visualization mechanisms used to supply information in Data Mining Environments, this work analyzes the visualization issue from a Human Computer Interaction setting. Important considerations arisen from the study of human perception are considered. Three practical examples based on numerical, textual and georeferenced data are described by means of the KNIME Analytics tool. In addition, the use and importance of graphs are emphasized for a correct information interpretation.

A Novel Privacy Preserving Data mining using improved decision tree and KP-ABE on High Dimensional Data

In distributed data mining environment maintaining individual data or patterns is a major issue due to high dimensionality and data size. Distributed Data mining framework can help to find the essential decision making patterns from distributed data. Privacy preserving data mining (PPDM) has emerged as a main research area for data confidentiality and knowledge sharing in between the communicating parties. As the distributed data of the individuals are stored by the third party, it leads to the misuse of distributed information in digital networks. Most of the decision patterns generated using the machine learning models for business organizations, industries and individuals has to be encoded before it is publicly shared or published. As the amount of data collected from different sources are increasing exponentially, the time taken to preserve the patterns using the traditional privacy preserving data mining models also increasing due to high computational attribute selection measures and noise in the distributed data. Also, filling sparse values using the conventional models are inefficient and infeasible for privacy preserving models. In this paper, a novel privacy preserving based classification model was designed and implemented on large datasets. In this model, a filter-based privacy preserving model using improved decision tree classifier is implemented to preserve the decision patterns using IPPDM-KPABE model. Experimental results proved that the proposed model has high computational efficiency compared to the traditional privacy preserving model on high dimensional datasets.

Frequent pattern sub-space clustering optimisation algorithm for data mining from large database

Data mining environment gives a quick response to the user by fast and correctly pick-out the item from the large database is a very challenging task. Previously, multiple algorithms were proposed to identify the frequent item since they are scanning database at multiple times. To overcome those problems, we proposed Rehashing based Apriori Technique in which hashing technology is used to store the data in horizontal and vertical formats. Rehash Based Apriori uses hashing function to reduce the size of candidate item set and scanning of database, eliminate non-frequent items and avoid hash collision. After finding frequent item sets, perform level wise subspace clustering. We instigate generalised self organised tree based (GSTB) mechanism to adaptively selecting root to construct the path from the cluster head to neighbours when constructing the tree. Our experimental results show that our proposed mechanisms reduce the computational time of overall process.

FREQUENT PATTERN SUB-SPACE CLUSTERING OPTIMIZATION (FPSSCO) ALGORITHM FOR DATAMINING FROM LARGE DATA BASE

Data mining environment gives a quick response to the user by fast and correctly pick-out the item from the large database is a very challenging task. Previously, multiple algorithms were proposed to identify the frequent item since they are scanning database at multiple times. To overcome those problems, we proposed Rehashing based Apriori Technique in which hashing technology is used to store the data in horizontal and vertical formats. Rehash Based Apriori uses hashing function to reduce the size of candidate item set and scanning of database, eliminate non-frequent items and avoid hash collision. After finding frequent item sets, perform level wise subspace clustering. We instigate generalised self organised tree based (GSTB) mechanism to adaptively selecting root to construct the path from the cluster head to neighbours when constructing the tree. Our experimental results show that our proposed mechanisms reduce the computational time of overall process.

A Survey of approaches for mining large data sets

Data mining environment creates a lot of information, that should be investigated, examples must be removed from that to pick up learning. In this new period with blast of information both organized and unstructured, in the field of genomics, meteorology, science, ecological examination and numerous others, it has gotten to be hard to process, oversee and break down examples utilizing customary databases and architectures. Thus, a legitimate structural engineering ought to be comprehended to pick up information about the Big Data.This paper presents a review of various algorithms from necessary for handling such large data set. These algorithms define various structures and methods implemented to handle Big Data, also in the paper are listed various tool that were developed for analyzing them.

Prioritization of anti-malarial hits from nature: chemo-informatic profiling of natural products with in vitro antiplasmodial activities and currently registered anti-malarial drugs.

BackgroundA large number of natural products have shown in vitro antiplasmodial activities. Early identification and prioritization of these natural products with potential for novel mechanism of action, desirable pharmacokinetics and likelihood for development into drugs is advantageous. Chemo-informatic profiling of these natural products were conducted and compared to currently registered anti-malarial drugs (CRAD).MethodsNatural products with in vitro antiplasmodial activities (NAA) were compiled from various sources. These natural products were sub-divided into four groups based on inhibitory concentration (IC50). Key molecular descriptors and physicochemical properties were computed for these compounds and analysis of variance used to assess statistical significance amongst the sets of compounds. Molecular similarity analysis, estimation of drug-likeness, in silico pharmacokinetic profiling, and exploration of structure–activity landscape were also carried out on these sets of compounds.ResultsA total of 1040 natural products were selected and a total of 13 molecular descriptors were analysed. Significant differences were observed among the sub-groups of NAA and CRAD for at least 11 of the molecular descriptors, including number of hydrogen bond donors and acceptors, molecular weight, polar and hydrophobic surface areas, chiral centres, oxygen and nitrogen atoms, and shape index. The remaining molecular descriptors, including clogP, number of rotatable bonds and number of aromatic rings, did not show any significant difference when comparing the two compound sets. Molecular similarity and chemical space analysis identified natural products that were structurally diverse from CRAD. Prediction of the pharmacokinetic properties and drug-likeness of these natural products identified over 50 % with desirable drug-like properties. Nearly 70 % of all natural products were identified as potentially promiscuous compounds. Structure–activity landscape analysis highlighted compound pairs that form ‘activity cliffs’. In all, prioritization strategies for the NAA were proposed.ConclusionsChemo-informatic profiling of NAA and CRAD have produced a wealth of information that may guide decisions and facilitate anti-malarial drug development from natural products. Articulation of the information provided within an interactive data-mining environment led to a prioritized list of NAA.Electronic supplementary materialThe online version of this article (doi:10.1186/s12936-016-1087-y) contains supplementary material, which is available to authorized users.

Interactive Document Clustering System Based on Coordinated Multiple Views

This paper proposes an interactive document clustering system, which is designed based on the concept of CMV (coordinated multiple views). An interactive document clustering is used by a user to obtain a set of document groups from a document collection in interactive manner. It is expected to be useful for various tasks such as text mining and document retrieval. As the result of document clustering consists of multiple objects such as clusters (document groups), documents, and words, each of those should be presented to users in different ways. Based on this consideration, the proposed system employs multiple views, each of which is designed for specific object such as document and keyword. A prototype system is implemented on TETDM (Total Environment for Text Data Mining), which is one of environments for developing text data mining tools. As it can provide the mechanism of coordination between modules, we decided to use it for developing the prototype system. The proposed system classifies information to be presented into 4 levels: clusters, document, bag of words, and word, each of which is displayed with different views. Experimental results with test participants show the effectiveness of the proposed system.

Scalable Predictive Analysis in Critically Ill Patients Using a Visual Open Data Analysis Platform.

With the accumulation of large amounts of health related data, predictive analytics could stimulate the transformation of reactive medicine towards Predictive, Preventive and Personalized (PPPM) Medicine, ultimately affecting both cost and quality of care. However, high-dimensionality and high-complexity of the data involved, prevents data-driven methods from easy translation into clinically relevant models. Additionally, the application of cutting edge predictive methods and data manipulation require substantial programming skills, limiting its direct exploitation by medical domain experts. This leaves a gap between potential and actual data usage. In this study, the authors address this problem by focusing on open, visual environments, suited to be applied by the medical community. Moreover, we review code free applications of big data technologies. As a showcase, a framework was developed for the meaningful use of data from critical care patients by integrating the MIMIC-II database in a data mining environment (RapidMiner) supporting scalable predictive analytics using visual tools (RapidMiner’s Radoop extension). Guided by the CRoss-Industry Standard Process for Data Mining (CRISP-DM), the ETL process (Extract, Transform, Load) was initiated by retrieving data from the MIMIC-II tables of interest. As use case, correlation of platelet count and ICU survival was quantitatively assessed. Using visual tools for ETL on Hadoop and predictive modeling in RapidMiner, we developed robust processes for automatic building, parameter optimization and evaluation of various predictive models, under different feature selection schemes. Because these processes can be easily adopted in other projects, this environment is attractive for scalable predictive analytics in health research.