Mining Metrics Research Articles

Quantifying chatbots’ ability to learn business processes

Chatbots enable organizations in the business-to-customer domain to respond to repetitive requests efficiently. Extant approaches in Natural Language Processing (NLP) already address the essential requirement of understanding user input and synthesizing a response as close as possible to a response a human interlocutor would give. However, we argue that the organizational adoption of chatbots further depends on the underlying model’s capability to learn and comply with organizations’ business processes, for example, authenticating a customer before providing sensitive details. To address this issue, we develop an approach that quantifies chatbots’ ability to learn business processes using standardized process mining metrics. We demonstrate our approach by training chatbots on a dataset of more than 500,000 customer service conversations from three companies on Twitter and show how our approach supports the quantification of a chatbot’s overall ability to learn business processes from the training data. Furthermore, we quantify a chatbot’s ability to learn a particular variant of the underlying process and we show how to compare the chatbot’s executed steps against a given normative process model. Our approach that seamlessly integrates with existing approaches to evaluate NLP-based chatbots mitigates the current hurdles that practitioners face and, therefore, strives to foster the adoption of chatbots in practice.

Rule-Based Models for Risk Estimation and Analysis of In-hospital Mortality in Emergency and Critical Care.

We propose a novel method that uses associative classification and odds ratios to predict in-hospital mortality in emergency and critical care. Manual mortality risk scores have previously been used to assess the care needed for each patient and their need for palliative measures. Automated approaches allow providers to get a quick and objective estimation based on electronic health records. We use association rule mining to find relevant patterns in the dataset. The odds ratio is used instead of classical association rule mining metrics as a quality measure to analyze association instead of frequency. The resulting measures are used to estimate the in-hospital mortality risk. We compare two prediction models: one minimal model with socio-demographic factors that are available at the time of admission and can be provided by the patients themselves, namely gender, ethnicity, type of insurance, language, and marital status, and a full model that additionally includes clinical information like diagnoses, medication, and procedures. The method was tested and validated on MIMIC-IV, a publicly available clinical dataset. The minimal prediction model achieved an area under the receiver operating characteristic curve value of 0.69, while the full prediction model achieved a value of 0.98. The models serve different purposes. The minimal model can be used as a first risk assessment based on patient-reported information. The full model expands on this and provides an updated risk assessment each time a new variable occurs in the clinical case. In addition, the rules in the models allow us to analyze the dataset based on data-backed rules. We provide several examples of interesting rules, including rules that hint at errors in the underlying data, rules that correspond to existing epidemiological research, and rules that were previously unknown and can serve as starting points for future studies.

Applying Learning Analytics to Detect Sequences of Actions and Common Errors in a Geometry Game.

Games have become one of the most popular activities across cultures and ages. There is ample evidence that supports the benefits of using games for learning and assessment. However, incorporating game activities as part of the curriculum in schools remains limited. Some of the barriers for broader adoption in classrooms is the lack of actionable assessment data, the fact that teachers often do not have a clear sense of how students are interacting with the game, and it is unclear if the gameplay is leading to productive learning. To address this gap, we seek to provide sequence and process mining metrics to teachers that are easily interpretable and actionable. More specifically, we build our work on top of Shadowspect, a three-dimensional geometry game that has been developed to measure geometry skills as well other cognitive and noncognitive skills. We use data from its implementation across schools in the U.S. to implement two sequence and process mining metrics in an interactive dashboard for teachers. The final objective is to facilitate that teachers can understand the sequence of actions and common errors of students using Shadowspect so they can better understand the process, make proper assessment, and conduct personalized interventions when appropriate.

Rules-based process mining to discover PLM system processes

The value of product lifecycle management systems (PLMS) is more and more recognised by companies and its use current has enormously increased. It is mainly used during the product design when different roles collaborate for sharing models, take review decisions, and approve or reject preliminary results. Often, companies have a general 'picture' about the processes involving PLMS (who performs an activity, when it is performed, what is done) but this knowledge can be reinforced, improved and modified using process mining. Here the knowledge is extracted from the event logs, and model-aware analytics are generated to evaluate the modelled, known and executed process. The business rules filter the logs and verify the impact on the process mining metrics to minimise the divergences between modelled and actual processes and improve the resulting quality metrics. The results help business users to identify lines of investigation for deviations from expected behaviour and propose improvement measures.

Rules-based process mining to discover PLM system processes

The value of product lifecycle management systems (PLMS) is more and more recognised by companies and its use current has enormously increased. It is mainly used during the product design when different roles collaborate for sharing models, take review decisions, and approve or reject preliminary results. Often, companies have a general 'picture' about the processes involving PLMS (who performs an activity, when it is performed, what is done) but this knowledge can be reinforced, improved and modified using process mining. Here the knowledge is extracted from the event logs, and model-aware analytics are generated to evaluate the modelled, known and executed process. The business rules filter the logs and verify the impact on the process mining metrics to minimise the divergences between modelled and actual processes and improve the resulting quality metrics. The results help business users to identify lines of investigation for deviations from expected behaviour and propose improvement measures.

Analysis of supply chain performance metrics for Indian mining & earthmoving equipment manufacturing companies using hybrid MCDM model

Companies with a high-performing supply chain (SC) often exhibit superior business performance. Therefore, companies are increasingly aligning their supply chain with business goals due to their impact on the firm's performance. Researchers acknowledge the importance of managing supply chain performance (SCP) in achieving greater customer experience and retention, increased cost-competitiveness, gaining market share with faster product innovation. Hence measuring and managing SCP is critical for a firm's business success. Though existing literature has explored key performance indicators (KPI) in measuring SCP but has rarely studied the interactions among the key SCP metrics and subsequent impact of overall SCP. To address the gap, present research work aimed to identify key metrics, rank them, investigate mutual relationship and propose integrated SCP metrics. Understanding the interrelationship among SCP metrics will enable supply chain practitioners in well-aligned decision making. To achieve this, a hybrid exploratory three-phased multi-criteria decision making (MCDM) model is adopted. The qualitative Delphi technique is initially applied to identify the critical SCP metrics, followed by Best-Worst Method (BWM) to rank SCP metrics, completed by decision making trial and evaluation laboratory (DEMATEL) method to establish the mutual relationship among the SCP metrics. The research finding, based on the study conducted for mining and earth-moving machinery manufacturing industry, suggests that sub-metrics such as flexibility in meeting customer needs, customer perceived value, operating cost of SC, employee productivity, range of products and services, are significantly contributing to the overall SCP. This study has proposed SCP metric selection model highlighting cross-functional and inter-organizational importance in performance management.

Algorithmic methods to explore the automation of the appraisal of structured and unstructured digital data

Purpose This paper aims to describe an interdisciplinary and innovative research conducted in Switzerland, at the Geneva School of Business Administration HES-SO and supported by the State Archives of Neuchâtel (Office des archives de l'État de Neuchâtel, OAEN). The problem to be addressed is one of the most classical ones: how to extract and discriminate relevant data in a huge amount of diversified and complex data record formats and contents. The goal of this study is to provide a framework and a proof of concept for a software that helps taking defensible decisions on the retention and disposal of records and data proposed to the OAEN. For this purpose, the authors designed two axes: the archival axis, to propose archival metrics for the appraisal of structured and unstructured data, and the data mining axis to propose algorithmic methods as complementary or/and additional metrics for the appraisal process. Design/methodology/approach Based on two axes, this exploratory study designs and tests the feasibility of archival metrics that are paired to data mining metrics, to advance, as much as possible, the digital appraisal process in a systematic or even automatic way. Under Axis 1, the authors have initiated three steps: first, the design of a conceptual framework to records data appraisal with a detailed three-dimensional approach (trustworthiness, exploitability, representativeness). In addition, the authors defined the main principles and postulates to guide the operationalization of the conceptual dimensions. Second, the operationalization proposed metrics expressed in terms of variables supported by a quantitative method for their measurement and scoring. Third, the authors shared this conceptual framework proposing the dimensions and operationalized variables (metrics) with experienced professionals to validate them. The expert’s feedback finally gave the authors an idea on: the relevance and the feasibility of these metrics. Those two aspects may demonstrate the acceptability of such method in a real-life archival practice. In parallel, Axis 2 proposes functionalities to cover not only macro analysis for data but also the algorithmic methods to enable the computation of digital archival and data mining metrics. Based on that, three use cases were proposed to imagine plausible and illustrative scenarios for the application of such a solution. Findings The main results demonstrate the feasibility of measuring the value of data and records with a reproducible method. More specifically, for Axis 1, the authors applied the metrics in a flexible and modular way. The authors defined also the main principles needed to enable computational scoring method. The results obtained through the expert’s consultation on the relevance of 42 metrics indicate an acceptance rate above 80%. In addition, the results show that 60% of all metrics can be automated. Regarding Axis 2, 33 functionalities were developed and proposed under six main types: macro analysis, microanalysis, statistics, retrieval, administration and, finally, the decision modeling and machine learning. The relevance of metrics and functionalities is based on the theoretical validity and computational character of their method. These results are largely satisfactory and promising. Originality/value This study offers a valuable aid to improve the validity and performance of archival appraisal processes and decision-making. Transferability and applicability of these archival and data mining metrics could be considered for other types of data. An adaptation of this method and its metrics could be tested on research data, medical data or banking data.

Multi-Core Processing Cloud Eclat Growth

Data mining is a lively process used in many leading technologies of this information era. Eclat growth is one of the best performance data mining algorithms. This work is indented to create a suave interface for Eclat growth algorithm to run in multi-core processor-based cloud computing environments. Recent improvements in processor manufacturing technology make it possible to create multi-core high performance Central Processing Units (CPUs) and Graphics Processing Units (GPUs). Many cloud services are already providing accessibility to these high-power processor virtual machines. The process of blending these technologies with Eclat Growth is proposed here in the name of “Multi-core Processing Cloud Eclat Growth” (MPCEG) to achieve higher processing speeds without compromising the standard data mining metrics such as Accuracy, Precision, Recall and F1-Score. New procedures for Cloud Parallel Processing, GPU Utilization, Annihilation of floating point arithmetic errors by fixed point replacement in GPUs and Hierarchical offloading aggregation are introduced in the construction process of proposed MPCEG.

Exhausting Agile Processing and Data Mining in Electronic Commerce

Agile software developments are hotspots of software development field in foreign countries. Metrics and big data will boost today's marketing leaders toward success. While the notion of the metrics-driven CMO is well-understood, marketers still struggle with how to apply agile and big data to deliver big business value. To do so successfully, they must get bigger their point of view further than promotional activities and center movements around their customers and their purchasing processes. Data mining metrics like this would give us great insight into whether the change was successful; whether it led to hairline fracture how it has changed user behavior, and whether it is delivering net value to the business in terms of the A/B test. The bottom few in the list could be used to track and improve it over time, considering there is some benefit to the business in terms of capturing the new data.

A Co-Training Strategy for Multiple View Clustering in Process Mining

Process mining refers to the discovery, conformance, and enhancement of process models from event logs currently produced by several information systems (e.g. workflow management systems). By tightly coupling event logs and process models, process mining makes it possible to detect deviations, predict delays, support decision making, and recommend process redesigns.Event logs are data sets containing the executions (called traces) of a business process. Several process mining algorithms have been defined to mine event logs and deliver valuable models (e.g. Petri nets) of how logged processes are being executed. However, they often generate spaghetti-like process models, which can be hard to understand. This is caused by the inherent complexity of real-life processes, which tend to be less structured and more flexible than what the stakeholders typically expect. In particular, spaghetti-like process models are discovered when all possible behaviors are shown in a single model as a result of considering the set of traces in the event log all at once.To minimize this problem, trace clustering can be used as a preprocessing step. It splits up an event log into clusters of similar traces, so as to handle variability in the recorded behavior and facilitate process model discovery. In this paper, we investigate a multiple view aware approach to trace clustering, based on a co-training strategy. In an assessment, using benchmark event logs, we show that the presented algorithm is able to discover a clustering pattern of the log, such that related traces result appropriately clustered. We evaluate the significance of the formed clusters using established machine learning and process mining metrics.

Sequence-based detection of sleeping cell failures in mobile networks

This article presents an automatic malfunction detection framework based on data mining approach to analysis of network event sequences. The considered environment is long term evolution (LTE) of Universal Mobile Telecommunications System with sleeping cell caused by random access channel failure. Sleeping cell problem means unavailability of network service without triggered alarm. The proposed detection framework uses N-gram analysis for identification of abnormal behavior in sequences of network events. These events are collected with minimization of drive tests functionality standardized in LTE. Further processing applies dimensionality reduction, anomaly detection with K-Nearest Neighbors, cross-validation, postprocessing techniques and efficiency evaluation. Different anomaly detection approaches proposed in this paper are compared against each other with both classic data mining metrics, such as F-score and receiver operating characteristic curves, and a newly proposed heuristic approach. Achieved results demonstrate that the suggested method can be used in modern performance monitoring systems for reliable, timely and automatic detection of random access channel sleeping cells.

Vaccine adverse event text mining system for extracting features from vaccine safety reports

To develop and evaluate a text mining system for extracting key clinical features from vaccine adverse event reporting system (VAERS) narratives to aid in the automated review of adverse event reports. Based upon clinical significance to VAERS reviewing physicians, we defined the primary (diagnosis and cause of death) and secondary features (eg, symptoms) for extraction. We built a novel vaccine adverse event text mining (VaeTM) system based on a semantic text mining strategy. The performance of VaeTM was evaluated using a total of 300 VAERS reports in three sequential evaluations of 100 reports each. Moreover, we evaluated the VaeTM contribution to case classification; an information retrieval-based approach was used for the identification of anaphylaxis cases in a set of reports and was compared with two other methods: a dedicated text classifier and an online tool. The performance metrics of VaeTM were text mining metrics: recall, precision and F-measure. We also conducted a qualitative difference analysis and calculated sensitivity and specificity for classification of anaphylaxis cases based on the above three approaches. VaeTM performed best in extracting diagnosis, second level diagnosis, drug, vaccine, and lot number features (lenient F-measure in the third evaluation: 0.897, 0.817, 0.858, 0.874, and 0.914, respectively). In terms of case classification, high sensitivity was achieved (83.1%); this was equal and better compared to the text classifier (83.1%) and the online tool (40.7%), respectively. Our VaeTM implementation of a semantic text mining strategy shows promise in providing accurate and efficient extraction of key features from VAERS narratives.

Mining metrics for buried treasure

The same but different: That might describe two metrics. On the surface CLASSI may show two metrics are locally equivalent, but buried beneath one may be a wealth of further structure. This was beautifully described in a paper by M.A.H. MacCallum in 1998. Here I will illustrate the effect with two flat metrics -- one describing ordinary Minkowski spacetime and the other describing a three-parameter family of Gal'tsov-Letelier-Tod spacetimes. I will dig out the beautiful hidden classical singularity structure of the latter (a structure first noticed by Tod in 1994) and then show how quantum considerations can illuminate the riches. I will then discuss how quantum structure can help us understand classical singularities and metric parameters in a variety of exact solutions mined from the Exact Solutions book.

Hierarchical Analysis for Discovering Knowledge in Large Databases

This article presents a goal-question-metric paradigm for selecting and implementing metrics for data mining, as well as a case study to illustrate that paradigm. the aim of this technique is to increase customer satisfaction and retention rate. It is especially useful for transaction records that are incomplete and therefore cannot be used as is for data mining. By analyzing the company's goal and strategies, data mining rules can be developed to discover associations within the available data.