Standard Process For Data Mining Research Articles

Application of CRISP-DM methodology for managing human-wildlife conflicts: an empirical case study in India

Human-wildlife conflict (HWC) is a major concern for protected area management. Managing HWC around protected areas requires structured and replicable processes to reduce subjectivity and promote adherence to good governance principles. The Cross-Industry Standard Process for Data Mining (CRISP-DM) is a widely-used process model for structured decision-making. This study demonstrates the novel application of CRISP-DM to HWC related decision-making. We apply CRISP-DM and conduct hotspot and temporal (monthly) analysis of HWC data from Ramnagar Forest Division, India. Based on the patterns of crop loss, livestock loss, and human loss, we propose conflict-type and species-specific preventive strategies. A qualitative assessment of the initial outcomes of the ongoing implementation finds the preventive strategies to be effective. We suggest a participatory approach, localization of strategy, and need for data management as opportunities for improvement.

Use of a business intelligence framework in the management of the quality of the electricity supply in small and medium-sized companies

This work shows a method based on Business Intelligence and its application for solving issues and problems in small and medium industries. In this matter, methodologies used by specialists were investigated: three methods were chosen and evaluated, and, of them, the CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology was used as a reference. Making an unconventional use of a commercial BI platform, the methodology was applied to a real case study: an agro-industrial company. After the application of the methodology and the descriptive, diagnostic and prescriptive analyzes, information and knowledge were found that allowed to propose the deployment of corrective measures that will improve the management of the company's electricity supply. This showed that these tools not only have application in commercial matters but also in the management of the electricity supply and, therefore, in the organizational performance of the company.

Knowledge Discovery in Engineering Applications Using Machine Learning Techniques

Abstract Sensorial acquired process data combined with machine learning (ML) algorithms are fundamental for mastering the challenges of modern production systems, however, their potential is rarely exploited in real-world manufacturing applications. In this context, the literature presents systematic procedure models to generate knowledge from data, such as the cross industry standard process for data mining (CRISP-DM) model, which is used as a standard methodology for conducting data mining in industrial applications. However, these models do not take into account boundary conditions of manufacturing processes as well as the characteristics of the sensorial acquired data within these systems to generate knowledge. Therefore, this work presents a novel procedure model for knowledge discovery in time series and image data in engineering applications (KDT-EA). A holistic view of knowledge discovery in manufacturing processes becomes feasible with a strong focus on data acquisition, data preprocessing, and data transformation to generate reliable input data for ML models estimating the actual state of manufacturing processes. The process model supports operators in industry setting up a suitable measurement chain acquiring high-quality data and selecting preparation techniques depending on superimposed disturbances. Furthermore, it suggests data transformation techniques reducing the amount of data without losing informational value and establishing a basis for product-related inline monitoring. To quantify the benefits of KDT-EA and the impact of its phase on the quality of the generated knowledge, the novel procedure model is applied to an application in the field of inline wear detection on a sheet metal forming tool.

Overcoming the pitfalls and perils of algorithms: A classification of machine learning biases and mitigation methods

Over the last decade, the importance of machine learning increased dramatically in business and marketing. However, when machine learning is used for decision-making, bias rooted in unrepresentative datasets, inadequate models, weak algorithm designs, or human stereotypes can lead to low performance and unfair decisions, resulting in financial, social, and reputational losses. This paper offers a systematic, interdisciplinary literature review of machine learning biases as well as methods to avoid and mitigate these biases. We identified eight distinct machine learning biases, summarized these biases in the cross-industry standard process for data mining to account for all phases of machine learning projects, and outline twenty-four mitigation methods. We further contextualize these biases in a real-world case study and illustrate adequate mitigation strategies. These insights synthesize the literature on machine learning biases in a concise manner and point to the importance of human judgment for machine learning algorithms.

Decision support system for fish quarantine measures in Indonesia

PurposeFish quarantine is a measure to prevent the entry and spread of quarantine fish pests and diseases abroad and from one area to another within Indonesia's territory. Based on these backgrounds, this study aims to identify the knowledge, knowledge management (KM) processes and knowledge management system (KMS) priority needs for quarantine fish and other fishery products measures (QMFFP) and then develop a classification model and web-based decision support system (DSS) for QMFFP decisions.Design/methodology/approachThis research methodology uses combination approaches, namely, contingency factor analysis (CFA), the cross-industry standard process for data mining (CRISP-DM) and knowledge management system development life cycle (KMSDLC). The CFA for KM solution design is performed by identifying KM processes and KMS priorities. The CRISP-DM for decision classification model is done by using a decision tree algorithm. The KMSDLC is used to develop a web-based DSS.FindingsThe highest priority requirements of KM technology for QMFFP are data mining and DSS with predictive features. The main finding of this study is to show that web-based DSS (functions and outputs) can support and accelerate QMFFP decisions by regulations and field practice needs. The DSS was developed using the CTree algorithm model, which has six main attributes and eight rules.Originality/valueThis study proposes a novel comprehensive framework for developing DSS (combination of CFA, CRISP-DM and KMSDLC), a novel classification model resulting from comparing two decision tree algorithms and a novel web-based DSS for QMFFP.

Modeling scientometric indicators using a statistical data ontology

Scientometrics is the field of study and evaluation of scientific measures such as the impact of research papers and academic journals. It is an important field because nowadays different rankings use key indicators for university rankings and universities themselves use them as Key Performance Indicators (KPI). The purpose of this work is to propose a semantic modeling of scientometric indicators using the ontology Statistical Data and Metadata Exchange (SDMX). We develop a case study at Tecnologico de Monterrey following the Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology. We evaluate the benefits of storing and querying scientometric indicators using linked data as a mean for providing flexible and quick access knowledge representation that supports indicator discovery, enquiring and composition. The semi-automatic generation and further storage of this linked data in the Neo4j graph database enabled an updatable and quick access model.

A Predictive Model for Network Intrusion Detection System Using Deep Neural Network

Network Intrusion Detection System (NIDS) is an important part of Cyber safety and security. It plays a key role in all networked ICT systems in detecting rampant attacks such as Denial of Service (DoS) and ransom ware attacks. Existing methods are inadequate in terms of accuracy detection of attacks. However, the requirement for high accuracy detection of attacks using Deep Neural Network requires expensive computing resources which in turn make most organisations, and individuals shy away from it. This study therefore aims at designing a predictive model for network intrusion detection using deep neural networks with very limited computing resources. The study adopted Cross Industry Standard Process for Data Mining (CRISP-DM) as one of the formal methodologies and python was used for both testing and training, using crucial parameters such as the learning rate, number of epochs, neurons and hidden layers which greatly determined the accuracy level of the DNN algorithm. These parameters were experimented with values that are lesser compared to previous studies, training and evaluation were also done on the KDD99 data-set. The varying values of accuracy obtained from this study on four models with different numbers of layers of 50-epochs and learning rate of 0.01 achieved competitive results in comparison with the previous research of 100-1000 epochs and learning rate of 0.1. Therefore, the model with two layers attained same accuracy of 0.955 as the model with three layers from the previous study out of the four models tested in this study.
 Also, the models with three and four layers in this study attained an accuracy of 0.956, which is 0.001greater than the previous study's models.
 Keywords: Network-Based IDS, Host-Based IDS, Deep Neural Network, Denial of Service, Knowledge Discovery Dataset

Implementasi Algoritma Naïve Bayes Classifier untuk Mendeteksi Berita Palsu pada Sosial Media

Hoax news (lie) on the internet has become a global problem that causes turmoil in society. Its presence can disrupt democratic order, the stability of social, cultural, political and economic life. The results of the research of the Indonesian Telematics Society showed that as many as 44.3% of respondents said they received fake news or misinformation every day. According to information released by Kominfo until August 11, 2021, there were 1848 hoax reports regarding the Covid-19 pandemic, 290 hoax reports regarding the Covid-19 Vaccine. Naïve Bayes Classifier is a classification method based on Bayes theorem, which in this paper is used to detect fake news on social media. The analysis was carried out using the Naïve Bayes Classifier algorithm, in this study using the CRISP-DM (Cross-Industry Standard Process for Data Mining) model. Training data sourced from the Kumparan site as much as 300 Data. In the process carried out using the python library for NLP, namely "<em>satrawi</em>". In testing the model using the confusion matrix method which consists of the number of rows of test data that are predicted to be true and false by the classification model used. At the deployment stage the model is pushed to Heroku so that users can predict news directly through the provided User Interface.

DEMONSTRATING HOW A HIGH- GROWTH FRAMEWORK COULD BE USED TO ASSIST A SOCIAL ENTERPRISE TO IDENTIFY GROWTH FACTORS AND IMPROVE SUSTAINABILITY

Social enterprises are critical players in economic growth and service delivery. This article presents a tool that helps to identify the aspects that affect the sustainability of social enterprises. The tool is based on a framework that identifies high-growth factors and that uses the CRISP-DM (Cross-industry standard process for data mining) methodology and mutual information analysis to develop a practical tool to support social enterprises’ viability and sustainability. The tool was applied to a case study to exemplify how practitioners and business owners could use it. In addition, this article shows how scholars could gain additional insights from data that had been previously analysed using a different methodology

Machine Learning Models for Predicting Financially Vigilant Low-Income Households

The COVID-19 pandemic has adversely affected households’ lives in terms of social and economic factors across the world. The Malaysian government has devised a number of stimulus packages to combat the pandemic’s effects. Stimulus packages would be insufficient to alleviate household financial burdens if they did not target those most affected by lockdowns. As a result, assessing household financial vigilance in the case of crisis like the COVID-19 pandemic is crucial. This study aimed to develop machine learning models for predicting and profiling financially vigilant households. The Special Survey on the Economic Effects of Covid-19 and Individual Round 1 provided secondary data for this study. As a research methodology, a cross-industry standard process for data mining is followed. Five machine learning algorithms were used to build predictive models. Among all, Gradient Boosted Tree was identified as the best predictive model based on F-score measure. The findings showed machine learning approach can provide a robust model to predict households’ financial vigilances, and this information might be used to build appropriate and effective economic stimulus packages in the future. Researchers, academics and policymakers in the field of household finance can use these recommendations to help them leverage machine learning.

The Sustainability Data Science Life Cycle for automating multi-purpose LCA workflows for the analysis of large product portfolios

Life Cycle Assessment (LCA) is a powerful and sophisticated tool to gain deep understanding of the environmental hotspots and optimization potentials of products. Yet, its cost-intensive manual data engineering and analysis workflows restrain its wider application in eco-design, green procurement, supply chain management, sustainable investment or other relevant business processes. Especially for large product portfolios and increasing reporting requirements, traditional LCA workflows and tools often fail to provide the necessary scalability. The Sustainability Data Science Life Cycle (S-DSLC) is a concept for workflow automation for multi-purpose LCA of large product portfolios. The concept integrates the frameworks of LCA, the cross-industry standard process for data mining (CRISP-DM), and the Data Science Life Cycle (DSLC). Key aspects of the concept are deep business-, stakeholder and user-understanding, deployment of LCA results in interactive browser tools (i.e. LCA-dashboards and Guided Analytics) tailored to the needs of individual roles and business processes, as well as the automation of data preparation, model generation and Life Cycle Impact Assessment based on modern data analytic tools. The demonstration of the concept shows substantial scalability improvements for dealing with large product portfolios and broad application of LCA results in various business processes.

Penerapan Algoritma C4.5 Pada Imbalanced Dataset Untuk Memprediksi Kegagalan Angsuran Properti

In this research, the data collection carried out by studying the patterns of consumers who fail to pay, which aimed to build a model so that it could be used in predicting customers who have the potential to fail to pay. The research used the Cross-Industry Standard Process for Data Mining (CRISP-DM) method with details of the business understanding process, data understanding, data preparation, modeling, evaluation and deployment / interpretation. The dataset in this research was taken from sales, cancellation and consumer data from January 2016 to December 2019. Because the dataset in this research was an imbalanced dataset, the researchers tried to use Synthetic Minority Oversampling Technique (SMOTE) in handling the imbalanced dataset. The research conducted a comparison of the value of accuracy, precision, recall, f measure and Area Under the ROC Curve (AUC) between the original dataset and the dataset for the addition of the SMOTE technique to several algorithms including C4.5, K-NN and Naïve Bayes. The attributes used in this research were source of funds, purpose of purchase, age, selling price, occupation, total installments, percentage of total installments, monthly installments, percentage of late installments and status. From the comparison, it was found that the C4.5 algorithm with the SMOTE 480% dataset had the highest accuracy value of 97.62%, precision of 0.976, recall of 0.976, f measure of 0.976 and AUC of 0.986 which meant Excellent Classification. From the research conducted, it was expected that the model formed on the imbalanced dataset with the C4.5 and SMOTE algorithms could be used to predict consumer installment failures.

The use of artificial neural networks and big data infrastructure for predictive analytics in solar energy

Abstract Renewable energy appears to be the solution to both the continuously growing energy demand, and pollution from fossil-based fuels. Recent advances in big data means that crucial areas of the energy supply chain are of interest to the use of advanced analytics. Solar energy is one of the most important renewable energy sources; however, it is stochastic, leading to production volatility and making it difficult to dispatch. The European Commission provides the legal framework and guidelines for increasing the adoption of renewable technologies in the European Union (EU). Meanwhile, the research community must provide solutions for increasing the predictability of solar energy: successful integration depends on how well solar energy production is predicted. Working under the Cross-Industry Standard Process for Data Mining, using real word operational data, this research focuses on providing a foundation of the analytics capabilities needed for reducing, or even removing, the disadvantages of solar energy, demonstrating that a world-class predicative tool can be obtained. Using weather and production data from photovoltaic cells installed in Romania, as a case study, coupled with the powerful artificial neural networks (ANN) architecture, results in a benchmark prediction performance. Currently, there is no research addressing photovoltaic energy production prediction by integrating the impact of artificial intelligence and big data.

Quality Improvement of NH1X36B Pre-Printed Box with QM-CRISP DM Approach at PT X

PT X is a manufacturer of cardboard box whose products are indispensable for various fields. The problem identified in NH1X36B pre-printed box, which is a shoes box, is the high defect rate that exceed company target of 2%. This study aims to reduce the defect rate of the product. The Quality-Management (QM) and Cross Industry Standard Process for Data Mining (CRISP-DM) approach was conducted by integrating Six Sigma and data mining. The Business Understanding phase was intended to define business and data mining objectives, SIPOC (Supplier-Input-Process-Output-Customer) diagram, and Critical-to-Quality (CTQ). In Data Understanding phase, it is known that the Defects Per Million Opportunities (DPMO) value is 1210.12. Data preparation phase was carried out with data cleaning, reduction, and discretization. Based on the Modeling result using Decision Tree C4.5 and FP-Growth algorithm, it is known that the dominant attributes causing high rejection are smeared ink, white spots, uneven varnish, and delamination. Decision Tree model accuracy of 90.24% indicates that the model is performing well. Analysis using FMEA yielded priority correction to the causes of smeared ink, uneven varnish, and delamination. Process improvement in Deployment phase was the application of plate cleaning and mounting form, printing process checklist, and SOP for sheet inspection. The improvement plans managed to improve the quality by rising sigma level from 4.533 to 4.648 sigma and decrease defect rate to 1.559%.

Development of glaucoma predictive model and risk factors assessment based on supervised models

ObjectivesTo develop and to propose a machine learning model for predicting glaucoma and identifying its risk factors.MethodData analysis pipeline is designed for this study based on Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology. The main steps of the pipeline include data sampling, preprocessing, classification and evaluation and validation. Data sampling for providing the training dataset was performed with balanced sampling based on over-sampling and under-sampling methods. Data preprocessing steps were missing value imputation and normalization. For classification step, several machine learning models were designed for predicting glaucoma including Decision Trees (DTs), K-Nearest Neighbors (K-NN), Support Vector Machines (SVM), Random Forests (RFs), Extra Trees (ETs) and Bagging Ensemble methods. Moreover, in the classification step, a novel stacking ensemble model is designed and proposed using the superior classifiers.ResultsThe data were from Shahroud Eye Cohort Study including demographic and ophthalmology data for 5190 participants aged 40-64 living in Shahroud, northeast Iran. The main variables considered in this dataset were 67 demographics, ophthalmologic, optometric, perimetry, and biometry features for 4561 people, including 4474 non-glaucoma participants and 87 glaucoma patients. Experimental results show that DTs and RFs trained based on under-sampling of the training dataset have superior performance for predicting glaucoma than the compared single classifiers and bagging ensemble methods with the average accuracy of 87.61 and 88.87, the sensitivity of 73.80 and 72.35, specificity of 87.88 and 89.10 and area under the curve (AUC) of 91.04 and 94.53, respectively. The proposed stacking ensemble has an average accuracy of 83.56, a sensitivity of 82.21, a specificity of 81.32, and an AUC of 88.54.ConclusionsIn this study, a machine learning model is proposed and developed to predict glaucoma disease among persons aged 40-64. Top predictors in this study considered features for discriminating and predicting non-glaucoma persons from glaucoma patients include the number of the visual field detect on perimetry, vertical cup to disk ratio, white to white diameter, systolic blood pressure, pupil barycenter on Y coordinate, age, and axial length.

Artificial intelligence for last-mile logistics - Procedures and architecture

Artificial Intelligence (AI) is becoming increasingly important in many industries due to its diverse areas of application and potential. In logistics in particular, increasing customer demands and the growth in shipment volumes are leading to difficulties in forecasting delivery times, especially for the last mile. This paper explores the potential of using AI to improve delivery forecasting. For this purpose, a structured theoretical solution approach and a method for improving delivery forecasting using AI are presented. In doing so, the important phases of the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework, a standard process for data mining, are adopted and discussed in detail to illustrate the complexity and importance of each task such as data preparation or evaluation. Subsequently, by embedding the described solution into an overall system architecture for information systems, ideas for the integration of the solution into the complexity of real information systems for logistics are given.

A data mining-based cross-industry process for predicting major bleeding in mechanical circulatory support.

Over a third of patients, treated with mechanical circulatory support (MCS) for end-stage heart failure, experience major bleeding. Currently, the prediction of a major bleeding in the near future is difficult because of many contributing factors. Predictive analytics using data mining could help calculating the risk of bleeding; however, its application is generally reserved for experienced researchers on this subject. We propose an easily applicable data mining tool to predict major bleeding in MCS patients. All data of electronic health records of MCS patients in the University Medical Centre Utrecht were included. Based on the cross-industry standard process for data mining (CRISP-DM) methodology, an application named Auto-Crisp was developed. Auto-Crisp was used to evaluate the predictive models for a major bleeding in the next 3, 7, and 30 days after the first 30 days post-operatively following MCS implantation. The performance of the predictive models is investigated by the area under the curve (AUC) evaluation measure. In 25.6% of 273 patients, a total of 142 major bleedings occurred during a median follow-up period of 542 [interquartile range (IQR) 205-1044] days. The best predictive models assessed by Auto-Crisp had AUC values of 0.792, 0.788, and 0.776 for bleedings in the next 3, 7, and 30 days, respectively. The Auto-Crisp-based predictive model created in this study had an acceptable performance to predict major bleeding in MCS patients in the near future. However, further validation of the application is needed to evaluate Auto-Crisp in other research projects.

Implementasi Metode Moving Average Sebagai Prediksi Penjualan Perlengkapan Pertanian Pada CV. Aneka Tani

CV. Aneka Tani is a shop that sells various agricultural equipment. The problems in CV. Aneka Tani are the process of recording transactions that are still done manually with a sales book and there is no sales prediction system. It causes waste of paper, human error in transactions, and bad stock management. The purpose of this research is to predict sales of agricultural equipment, applying the Moving Average algorithm to CV. Aneka Tani’s data to generate sales prediction models, and analyze predictive models. The research method used is the CRISP-DM (Cross Industry Standard Process for Data Mining) which consists of business understanding, data understanding, data preparation, modeling, evaluation, and implementation. The process of collecting data is done by conducting interviews with business owners. There were 6 products used in Moving Average method. The stable product is Dafat with MAD value is 0,9 and MSE value is 1,2. The non stabel product is Phonska with MAD value is 13,6 and MSE value is 245,7.

A Study on Singapore’s Ageing Population in the Context of Eldercare Initiatives Using Machine Learning Algorithms

Ageing has always directly impacted the healthcare systems and, more specifically, the eldercare costs, as initiatives related to eldercare need to be addressed beyond the regular healthcare costs. This study aims to examine the general issues of eldercare in the Singapore context, as the population of the country is ageing rapidly. The main objective of the study is to examine the eldercare initiatives of the government and their likely impact on the ageing population. The methodology adopted in this study is Cross-Industry Standard Process for Data Mining (CRISP-DM). Reviews related to the impact of an ageing population on healthcare systems in the context of eldercare initiatives were studied. Analysis methods include correlation and machine learning algorithms, such as Decision Tree, Logistic Regression and Receiver Operating Characteristics curve analysis. Suggestions have been provided for various healthcare and eldercare systems’ initiatives and needs that are required to transform to cope with the ageing population.

Automated Business Goal Extraction from E-mail Repositories to Bootstrap Business Understanding

The Cross-Industry Standard Process for Data Mining (CRISP-DM), despite being the most popular data mining process for more than two decades, is known to leave those organizations lacking operational data mining experience puzzled and unable to start their data mining projects. This is especially apparent in the first phase of Business Understanding, at the conclusion of which, the data mining goals of the project at hand should be specified, which arguably requires at least a conceptual understanding of the knowledge discovery process. We propose to bridge this knowledge gap from a Data Science perspective by applying Natural Language Processing techniques (NLP) to the organizations’ e-mail exchange repositories to extract explicitly stated business goals from the conversations, thus bootstrapping the Business Understanding phase of CRISP-DM. Our NLP-Automated Method for Business Understanding (NAMBU) generates a list of business goals which can subsequently be used for further specification of data mining goals. The validation of the results on the basis of comparison to the results of manual business goal extraction from the Enron corpus demonstrates the usefulness of our NAMBU method when applied to large datasets.