Data Science Tasks Research Articles

A novel and efficient risk minimisation-based missing value imputation algorithm

Missing value imputation (MVI) is a key task in data science, in which learning models are built from incomplete data. In contrast to externally driven MVI algorithms, this study proposes a novel risk minimisation-based MVI algorithm (RM-MVI) that considers both the internal characteristics of missing data and the external performance for specific classification applications. RM-MVI is technically designed for labelled data and is applied in two stages: filling with structural risk minimisation (SRM) and refining with empirical risk minimisation (ERM). In the filling stage, an autoencoder with a single hidden layer is trained on the original dataset without missing values. Missing values are first initialised with random numbers, and the imputation values are then preliminarily optimised based on the derived updating rule to minimise the structural risk-oriented objective function. After the imputation values have been preliminarily optimised in the filling stage, a neural-network-based classifier is trained in the refining stage to optimise the imputation values sophisticatedly by reducing the empirical risk. Experiments were conducted on several benchmark datasets to validate the feasibility, rationality, and effectiveness of the proposed RM-MVI algorithm. The results show that (1) the optimisation processes of the imputation values corresponding to the SRM and ERM are convergent so that the optimised imputation values can be obtained; (2) SRM can ensure distribution consistency of the imputation values that are preliminarily optimised in the filling stage, while ERM can optimise the imputation values sophisticatedly in the refining stage, which is more helpful for classifier training; and (3) the RM-MVI algorithm can yield considerably better MVI performance on benchmark datasets than 11 well-known MVI algorithms, such as a 26% higher distribution consistency ratio and 2% to 5% higher testing accuracies for 6 classifiers on average. This demonstrates that RM-MVI is a viable approach for addressing MVI problems.

Inside Production Data Science: Exploring the Main Tasks of Data Scientists in Production Environments

Modern production relies on data-based analytics for the prediction and optimization of production processes. Specialized data scientists perform tasks at companies and research institutions, dealing with real data from actual production environments. The roles of data preprocessing and data quality are crucial in data science, and an active research field deals with methodologies and technologies for this. While anecdotes and generalized surveys indicate preprocessing is the major operational task for data scientists, a detailed view of the subtasks and the domain of production data is missing. In this paper, we present a multi-stage survey on data science tasks in practice in the field of production. Using expert knowledge and insights, we found data preprocessing to be the major part of the tasks of data scientists. In detail, we found that tackling missing values, finding data point meanings, and synchronization of multiple time-series were often the most time-consuming preprocessing tasks.

SAGES surgical data science task force: enhancing surgical innovation, education and quality improvement through data science.

SAGES surgical data science task force: enhancing surgical innovation, education and quality improvement through data science.

Determining Exact Quantiles with Randomized Summaries

Quantiles are fundamental statistics in various data science tasks, but costly to compute, e.g., by loading the entire data in memory for ranking. With limited memory space, prevalent in end devices or databases with heavy loads, it needs to scan the data in multiple passes. The idea is to gradually shrink the range of the queried quantile till it is small enough to fit in memory for ranking the result. Existing methods use deterministic sketches to determine the exact range of quantile, known as deterministic filter, which could be inefficient in range shrinking. In this study, we propose to shrink the ranges more aggressively, using randomized summaries such as KLL sketch. That is, with a high probability the quantile lies in a smaller range, namely probabilistic filter, determined by the randomized sketch. Specifically, we estimate the expected passes for determining the exact quantiles with probabilistic filters, and select a proper probability that can minimize the expected passes. Analyses show that our exact quantile determination method can terminate in P passes with 1-δ confidence, storing O(N 1/P logP-1/2P (1/δ)) items, close to the lower bound Ømega(N1/P) for a fixed δ. The approach has been deployed as a function in an LSM-tree based time-series database Apache IoTDB. Remarkably, the randomized sketches can be pre-computed for the immutable SSTables in LSM-tree. Moreover, multiple quantile queries could share the data passes for probabilistic filters in range estimation. Extensive experiments on real and synthetic datasets demonstrate the superiority of our proposal compared to the existing methods with deterministic filters. On average, our method takes 0.48 fewer passes and 18% of the time compared with the state-of-the-art deterministic sketch (GK sketch).

Exploring the roles of AI-Assisted ChatGPT in the field of data science

In this study, we explore the roles of AI-assisted ChatGPT (Generative Pre-trained Transformer) in the field of data science. AI-assisted ChatGPT, a powerful language model, is fine-tuned using domain-specific data for specialised data science tasks, such as sentiment analysis and named entity recognition (NER). The results reveal significant reductions in model size and memory usage with minor trade-offs in inference time, providing valuable resource-efficient deployment. Various data augmentation methods, including back-translation, synonym replacement, and contextual word embeddings, are employed to augment the training dataset. The study's results are subjected to rigorous statistical analysis, including paired t-tests and ANOVA tests, to determine the significance of the findings. The research concludes with insightful suggestions and future scope, including advanced fine-tuning strategies, model optimization techniques, and ethical considerations.

Coding Code: Qualitative Methods for Investigating Data Science Skills

Despite the elevated importance of Data Science in Statistics, there exists limited research investigating how students learn the computing concepts and skills necessary for carrying out data science tasks. Computer Science educators have investigated how students debug their own code and how students reason through foreign code. While these studies illuminate different aspects of students’ programming behavior or conceptual understanding, a method has yet to be employed that can shed light on students’ learning processes. This type of inquiry necessitates qualitative methods, which allow for a holistic description of the skills a student uses throughout the computing code they produce, the organization of these descriptions into themes, and a comparison of the emergent themes across students or across time. In this article we share how to conceptualize and carry out the qualitative coding process with students’ computing code. Drawing on the Block Model to frame our analysis, we explore two types of research questions which could be posed about students’ learning. Supplementary materials for this article are available online.

Oracle-Preserving Latent Flows

A fundamental task in data science is the discovery, description, and identification of any symmetries present in the data. We developed a deep learning methodology for the simultaneous discovery of multiple non-trivial continuous symmetries across an entire labeled dataset. The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function, ensuring the desired symmetry properties. The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to invariant transformations with respect to high-dimensional oracles. The method is demonstrated with several examples on the MNIST digit dataset, where the oracle is provided by the 10-dimensional vector of logits of a trained classifier. We find classes of symmetries that transform each image from the dataset into new synthetic images while conserving the values of the logits. We illustrate these transformations as lines of equal probability (“flows”) in the reduced latent space. These results show that symmetries in the data can be successfully searched for and identified as interpretable non-trivial transformations in the equivalent latent space.

Discovering High Utility Episodes in Sequences

Sequence data is more commonly seen than other types of data (e.g., transaction data) in real-world applications. For the mining task from sequence data, several problems have been formulated, such as sequential pattern mining, episode mining, and sequential rule mining. As one of the fundamental problems, episode mining has often been studied. The common wisdom is that discovering frequent episodes is not useful enough. In this paper, we propose an efficient utility mining approach, namely UMEpi: <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</u> tility <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> ining of high-utility <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Epi</u> sodes from complex event sequences. We propose the concept of remaining utility of episodes and achieve a tighter upper bound, namely episode-weighted utilization ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> ), which will provide better pruning. Thus, the optimized <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> -based pruning strategies can achieve better improvements in mining efficiency. The search space of UMEpi w.r.t. a prefix-based lexicographic sequence tree is spanned and determined recursively for mining high-utility episodes, by prefix-spanning in a depth-first way. Finally, extensive experiments on four real-life datasets demonstrate that UMEpi can discover the complete high-utility episodes from complex event sequences. Furthermore, the improved variants of UMEpi significantly outperform the baseline in terms of execution time, memory consumption, and scalability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement</i> —This article contributes to the problem of utility-based episode discovery, which is a challenging task in artificial intelligence and data science. To the best of our knowledge, it is the first article that introduces the concept of the remaining utility of an episode in an event sequence and then formulates an alternative definition of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> , which is able to provide an accurate formulation of the upper bound. The designed UMEpi algorithm can be a benchmark for utility-based episode discovery. UMEpi addresses several challenges, and the optimized <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EWU</i> -based pruning strategies can achieve better improvements in mining efficiency on massive datasets. It is well known that episode discovery has been successfully applied in a wide range of real-world applications and various domains. In summary, this correct and efficient utility episode mining algorithm contributes to the artificial intelligence systems in many applications, especially complex event processing and analysis.

Data Preprocessing Toolkit : An Approach to Automate Data Preprocessing

-Data Pre-processing transforms the data into a format that is more easily and effectively processed in data mining, machine learning, and other data science tasks. The Data Pre-processing techniques are generally used at the earlier stages of the machine learning and AI development pipeline to ensure accurate results. Data must first go through several pre-processing steps before the machine learning model can use it. We intend to build a GUI that allows users to input the data with inconsistencies and various options are provided to pre- process. After the data is processed, we provide the users an option to analyze the data which gives them clarity on the further aspects of preprocessing. Pre-processing consists of three main phases i.e. Data Cleaning, Data Transformation, and Data Reduction. Data cleaning involves correcting or removing inaccurate, corrupted, improperly formatted, duplicate, or incomplete data from a dataset. Data transformation is a technique used to convert data into a suitable format that eases data mining and retrieves strategic information. Data reduction is the process of shrinking the size of the original data so that it can be represented in a much smaller volume. Preprocessing requires a lot of work, what if we could automate it? With just one click, this web application is capable of transforming inconsistent data into consistent data that could be used further. The GUI is built using the Streamlit library in python and in the backend, we intend to use the AutoClean library along with many other libraries based on different formats of data.. Key Words: Data Cleaning, AutoClean, Data Transformation, Data Reduction, Streamlit.

How sustainable is “common” data science in terms of power consumption?

Continuous developments in data science have brought forth an exponential increase in complexity of machine learning models. Additionally, data scientists have become ubiquitous in the private market and academic environments. All of these trends are on a steady rise, and are associated with an increase in power consumption and associated carbon footprint. The increasing carbon footprint of large-scale advanced data science has already received attention, but the latter trend has not. This work aims to estimate the contribution of the increasingly popular “common” data science to the global carbon footprint. To this end, the power consumption of several typical tasks in the aforementioned common data science tasks are measured and compared to: large-scale “advanced” data science, common computer-related tasks, and everyday non-computer related tasks. An automated data science project is also run on various hardware architectures. To assess its sustainability in terms of carbon emission, the measurements are converted to gCO2eq and an equivalent unit of “km driven by car”. Our main findings are: “common” data science consumes 2.57 more power than regular computer usage, but less than some common everyday power-consuming tasks such as lighting or heating; advanced data science consumes substantially more power than common data science, and can be either on par or vastly surpass common everyday power-consuming tasks, depending on the scale of the project. In addition to the reporting of these results, this work also aims to inspire researchers to include power usage and estimated carbon emission as a secondary result in their work.

US-Rule: Discovering Utility-driven Sequential Rules

Utility-driven mining is an important task in data science and has many applications in real life. High-utility sequential pattern mining (HUSPM) is one kind of utility-driven mining. It aims at discovering all sequential patterns with high utility. However, the existing algorithms of HUSPM can not provide a relatively accurate probability to deal with some scenarios for prediction or recommendation. High-utility sequential rule mining (HUSRM) is proposed to discover all sequential rules with high utility and high confidence. There is only one algorithm proposed for HUSRM, which is not efficient enough. In this article, we propose a faster algorithm called US-Rule, to efficiently mine high-utility sequential rules. It utilizes the rule estimated utility co-occurrence pruning strategy (REUCP) to avoid meaningless computations. Moreover, to improve its efficiency on dense and long sequence datasets, four tighter upper bounds (LEEU, REEU, LERSU, and RERSU) and corresponding pruning strategies (LEEUP, REEUP, LERSUP, and RERSUP) are designed. US-Rule also proposes the rule estimated utility recomputing pruning strategy (REURP) to deal with sparse datasets. Finally, a large number of experiments on different datasets compared to the state-of-the-art algorithm demonstrate that US-Rule can achieve better performance in terms of execution time, memory consumption, and scalability.

Formula omitted]-HMM and optimal decoding higher-order structures on sequential data

Decoding higher-order structure on sequential data is an indispensable task in data science. It requires models to have the capability to characterize interdependencies among hidden events that have generated observable data. However, to be able to decode arbitrary structures, such models would need to cope with the intractability arising from computing context-sensitive relations, likely compromising the quality of answers. To address this important issue, the current paper introduces the arbitrary order hidden Markov model (α-HMM), an extension of the HMM that permits decoding of the optimal higher-order structure with an assurance of computational tractability. The advantage of the α-HMMis made possible by an identified principle on how random variables influence each other in a stochastic process. In particular, it is shown that decoding the optimal structure with an α-HMMcan be computed in O(n3)-time for any stochastic process of n random variables. As an application, it is demonstrated the decoding algorithm inspires a simple yet effective algorithm for RNA secondary structure prediction.

On Sense Making and the Generation of Knowledge in Visual Analytics

Interactive visual tools and related visualization technologies, built to support explorative data analysis, ultimately lead to sense making and knowledge discovery from large volumes of raw data. These processes namely rely on human visual perception and cognition, in which human analysts perceive external representations (system structure, dataset, integral data visualizations) and form respective internal representations (internal cognitive imprints of external systems) that enable deeper comprehension of the employed system and the underlying data features. These internal representations further evolve through continuous interaction with external representations. They also depend on the individual’s own cognitive pathways. Currently, there has been insufficient work on understanding how these internal cognitive mechanisms form and function. Hence, we aim to offer our own interpretations of such processes observed through our daily data exploration workflows. This is accomplished by following specific explorative data science tasks while working with diverse interactive visual systems and related notebook style environments that have different organizational structures and thus may entail different approaches to thinking and shaping sense making and knowledge generation. In this paper, we deliberate on the cognitive implications for human analysists when interacting with such a diverse organizational structure of tools and approaches when performing the essential steps of an explorative visual analysis.

Next-generation energy systems for sustainable smart cities: Roles of transfer learning

Smart cities attempt to reach net-zero emissions goals by reducing wasted energy while improving grid stability and meeting service demand. This is possible by adopting next-generation energy systems, which leverage artificial intelligence, the Internet of things (IoT), and communication technologies to collect and analyze big data in real-time and effectively run city services. However, training machine learning algorithms to perform various energy-related tasks in sustainable smart cities is a challenging data science task. These algorithms might not perform as expected, take much time in training, or do not have enough input data to generalize well. To that end, transfer learning (TL) has been proposed as a promising solution to alleviate these issues. To the best of the authors’ knowledge, this paper presents the first review of the applicability of TL for energy systems by adopting a well-defined taxonomy of existing TL frameworks. Next, an in-depth analysis is carried out to identify the pros and cons of current techniques and discuss unsolved issues. Moving on, two case studies illustrating the use of TL for (i) energy prediction with mobility data and (ii) load forecasting in sports facilities are presented. Lastly, the paper ends with a discussion of the future directions.

DORIAN in action

Existing automated machine learning solutions and intelligent discovery assistants are popular tools that facilitate the end-user with the design of data science (DS) pipelines. However, they yield limited applicability for a wide range of real-world use cases and application domains due to (a) the limited support of DS tasks; (b) a small, static set of available operators; and (c) restriction to evaluation processes with quantifiable loss functions. We demonstrate DORIAN, a human-in-the-loop approach for the assisted design of data science pipelines that supports a large and growing set of DS tasks, operators, and arbitrary user-defined evaluation processes. Based on the user query, i.e., a dataset and a DS task, DORIAN computes a ranked list of candidate pipelines that the end-user can choose from, alter, execute and evaluate. It stores executed pipelines in an experiment database and utilizes similarity-based search to identify relevant previously-run pipelines from the experiment database. DORIAN also takes user interaction into account to improve suggestions over time. We show how users can interact with DORIAN to create and compare DS pipelines on various real-world DS tasks without the need for writing any code.

Demonstration of collaborative and interactive workflow-based data analytics in texera

Collaborative data analytics is becoming increasingly important due to the higher complexity of data science, more diverse skills from different disciplines, more common asynchronous schedules of team members, and the global trend of working remotely. In this demo we will show how Texera supports this emerging computing paradigm to achieve high productivity among collaborators with various backgrounds. Based on our active joint projects on the system, we use a scenario of social media analysis to show how a data science task can be conducted on a user friendly yet powerful platform by a multi-disciplinary team including domain scientists with limited coding skills and experienced machine learning experts. We will present how to do collaborative editing of a workflow and collaborative execution of the workflow in Texera. We will focus on data-centric features such as synchronization of operator schemas among the users during the construction phase, and monitoring and controlling the shared runtime during the execution phase.

A Stochastic Approximation-Langevinized Ensemble Kalman Filter Algorithm for State Space Models with Unknown Parameters

Inference for high-dimensional, large scale and long series dynamic systems is a challenging task in modern data science. The existing algorithms, such as particle filter or sequential importance sampler, do not scale well to the dimension of the system and the sample size of the dataset, and often suffers from the sample degeneracy issue for long series data. The recently proposed Langevinized ensemble Kalman filter (LEnKF) addresses these difficulties in a coherent way. However, it cannot be applied to the case that the dynamic system contains unknown parameters. This paper proposes the so-called stochastic approximation-LEnKF for jointly estimating the states and unknown parameters of the dynamic system, where the parameters are estimated on the fly based on the state variables simulated by the LEnKF under the framework of stochastic approximation Markov chain Monte Carlo (MCMC). Under mild conditions, we prove its consistency in parameter estimation and ergodicity in state variable simulations. The proposed algorithm can be used in uncertainty quantification for long series, large scale, and high-dimensional dynamic systems. Numerical results indicate its superiority over the existing algorithms. We employ the proposed algorithm in state-space modeling of the sea surface temperature with a long short term memory (LSTM) network, which indicates its great potential in statistical analysis of complex dynamic systems encountered in modern data science. A supplement to the paper is available online.

Genetic Programming-Based Feature Construction for System Setting Recognition and Component-Level Prognostics

Extracting representative feature sets from raw signals is crucial in Prognostics and Health Management (PHM) for components’ behavior understanding. The literature proposes various methods, including signal processing in the time, frequency, and time–frequency domains, feature selection, and unsupervised feature learning. An emerging task in data science is Feature Construction (FC), which has the advantages of both feature selection and feature learning. In particular, the constructed features address a specific objective function without requiring a label during the construction process. Genetic Programming (GP) is a powerful tool to perform FC in the PHM context, as it allows to obtain distinct feature sets depending on the analysis goal, i.e., diagnostics and prognostics. This paper adopts GP to extract system-level features for machinery setting recognition and component-level features for prognostics. Three distinct fitness functions are considered for the GP training, which requires a set of statistical time-domain features as input. The methodology is applied to vibration signals extracted from a test rig during run-to-failure tests under different settings. The performances of constructed features are evaluated through the classification accuracy and the Remaining Useful Life (RUL) prediction error. Results demonstrate that GP-based features classify known and novel machinery operating conditions better than feature selection and learning methods.

Global Building Morphology Indicators

Characterising and analysing urban morphology is a continuous task in urban data science, environmental analyses, and many other domains. As the availability and quality of data on them have been increasing, buildings have gained more attention. However, tools and data facilitating large-scale studies, together with an interdisciplinary consensus on metrics, remain scarce and often inadequate. We present Global Building Morphology Indicators (GBMI) — a three-pronged contribution addressing such shortcomings: (i) a comprehensive list of hundreds of building form multi-scale measures derived through a systematic literature review; (ii) a methodology and tool for the computation of these metrics in a database suited for big data and comparative studies, and release the code freely and open-source; and (iii) we carry out the computations using high performance computing, generating a public repository with data quantifying the form of selected urban areas around the world, and demonstrate their value with novel analyses comparing morphological parameters across cities. GBMI introduces a formalised, structured, modular, and extensible method to compute, manage, and disseminate urban indicators at a large scale and high resolution, while the precomputed dataset facilitates comparative studies. The theory and implementation traverse multiple scales: at the building level, both individual and contextual ones based on encircling buildings by multiple buffers, and aggregations at several hierarchical administrative levels and at multiple grids. Our open dataset, comprising billions of records on a growing scope of urban areas worldwide, is the most comprehensive instance of morphological data parametrising the individual building stock, supporting studies in urban analytics and a range of disciplines.

An educator’s perspective of the tidyverse

Computing makes up a large and growing component of data science and statistics courses. Many of those courses, especially when taught by faculty who are statisticians by training, teach R as the programming language. A number of instructors have opted to build much of their teaching around use of the tidyverse. The tidyverse, in the words of its developers, "is a collection of R packages that share a high-level design philosophy and low-level grammar and data structures, so that learning one package makes it easier to learn the next". These shared principles have led to the widespread adoption of the tidyverse ecosystem. A large part of this usage is because the tidyverse tools have been intentionally designed to ease the learning process and make it easier for users to learn new functions as they engage with additional pieces of the larger ecosystem. Moreover, the functionality offered by the packages within the tidyverse spans the entire data science cycle, which includes data import, visualisation, wrangling, modeling, and communication. We believe the tidyverse provides an effective and efficient pathway for undergraduate students at all levels and majors to gain computational skills and thinking needed throughout the data science cycle. In this paper, we introduce the tidyverse from an educator's perspective. We provide a brief introduction to the tidyverse, demonstrate how foundational statistics and data science tasks are accomplished with the tidyverse, and discuss the strengths of the tidyverse, particularly in the context of teaching and learning.