Data Science Problems Research Articles

BILEVEL OPTIMIZED RECURSIVE FEATURE ELIMINATOR FOR CERVICAL CANCER FEATURE SELECTION PROCESS

An immense need has emerged in several areas of biological area for the development of prediction algorithms capable of managing the increasing complexity of high-dimensional information. In developing countries, Cervical Cancer (CC) kills more women than any other disease or accident, and it's the top cause of death among women worldwide. Early detection and treatment lead to improved results and longer patient survival, which in turn reduces cancer mortality. For the majority of real-world data science problems, not all dataset variables are useful for building models. The accuracy of a classifier and the model's ability to generalize are both reduced by repeated variables. Furthermore, adding more variables increases the overall complexity of a model. Deep learning's feature selection approach is a good fit for this issue. When it comes to selecting features for linear regression, our novel Bilevel Optimized Recursive Feature Eliminator (BORFE) method represents a revolutionary development. Finding the optimal fit for a model and eliminating its most undesirable features is the objective of this innovative feature selection method. This study presents a novel cross-validation approach using a Bilevel Optimization–Based -Recursive Feature Extractor (BORFE) to perform an in-depth analysis of the hyper-parameters. When used with cross-validation, RFE finds the optimal number of features and the optimum selection of ranking features. According to the evaluation metrics, BORFE performs better than the other conventional algorithms when it comes to FS on cervical cancer datasets. For the cervical cancer dataset, this shows that BORFE can solve FS problems successfully.

Taxonomic Data Objects for Communicating the Meaning of Species Names

A central problem in biodiversity data science remains the inability to precisely aggregate observations that originate from the same species. Current approaches still rely heavily on the ‘Genus species’ pair of Linnaean taxonomy to label and group data. However, such binomial species names do not alone contain enough information to distinguish variation in a name’s conceptual usage (i.e., meaning) across time, place, or investigator. This “species name-to-meaning” problem is well known, but no robust and scalable solutions yet exist (Berendsohn 1995Bisby 2000, Sutherland et al. 2000, Page 2007, Franz and Sterner 2018, Upham et al. 2021, Sterner et al. 2023, Sterner et al. 2020, Beach et al. 1993). The problem is widespread especially for well-studied groups like mammals, for which 45% more species are now recognized than 30 years ago, including many species splits and rearrangements (Burgin et al. 2018, Mammal Diversity Database 2024). To address this problem, we here propose to digitally package and sign the relevant taxonomic data associated with a given species name usage to form a ‘Taxonomic Data Object’ (TDO). These TDOs are conceptually similar to 'secundus' references (i.e., species name sec. author, Berendsohn 1995) but differ in being machine readable and thus operational from the perspective of high-throughput data science. TDOs aim to move from analog secundus references, which require a human to track meaning by reading the cited article, to a digital and machine-actionable taxonomic reference. Versioned TDOs that are digitally signed using hash algorithms (e.g., md5 or sha256, see Elliott et al. 2023) will allow for precisely communicating, with verified data provenance from sender to receiver, certain aspects of what a species name means according to a given authority at a given time. Example TDO contents include DarwinCore terms (e.g., scientificName, namePublishedIn, nameAccordingTo, nomenclaturalStatus (Darwin Core Maintenance Group 2021)) as well as meaning-rich digital assets like geographic range maps (e.g., GeoJSON format), exemplar DNA sequences (e.g., FASTA format or National Center for Biotechnology Information (NCBI) accession number), holotype specimen information (e.g., catalog number, type locality coordinates), and taxonomic treatment texts (including material citations, e.g., as digitized and extracted by Plazi TreatmentBank, Agosti and Egloff 2009; see Fig. 1). We discuss pilot examples comparing global bat species according to the v1.2 vs. v1.11 taxonomies of the Mammal Diversity Database (Sep 2020 vs. Apr 2023), showing how TDOs enable the reliable tracking of taxonomic meaning for the 63 bat species affected by splits during this period. We posit that widespread use of TDOs will enable the traceable exchange of taxonomic information across a variety of existing platforms, providing a path for accurate species-level data aggregation at global scales, at least for well-studied taxa.

An information-theoretic framework for conditional causality analysis of brain networks.

Identifying directed network models for multivariate time series is a ubiquitous problem in data science. Granger causality measure (GCM) and conditional GCM (cGCM) are widely used methods for identifying directed connections between time series. Both GCM and cGCM have frequency-domain formulations to characterize the dependence of time series in the spectral domain. However, the original methods were developed using a heuristic approach without rigorous theoretical explanations. To overcome the limitation, the minimum-entropy (ME) estimation approach was introduced in our previous work (Ning & Rathi, 2018) to generalize GCM and cGCM with more rigorous frequency-domain formulations. In this work, this information-theoretic framework is further generalized with three formulations for conditional causality analysis using techniques in control theory, such as state-space representations and spectral factorizations. The three conditional causal measures are developed based on different ME estimation procedures that are motivated by equivalent formulations of the classical minimum mean squared error estimation method. The relationship between the three formulations of conditional causality measures is analyzed theoretically. Their performance is evaluated using simulations and real neuroimaging data to analyze brain networks. The results show that the proposed methods provide more accurate network structures than the original approach.

Multi-factor forecasting of statistical trends for data science problems

The article deals with the processes of multi-factor forecasting of statistical trends for Data Science problems. Most of the classic approaches to data processing consist of studying the consequences of phenomena rather than the factors of their appearance. At the same time, the factors affecting the behavior of the investigated process are assumed to be random and are not investigated. The article discusses the approach to forecasting the parameters of the trend of statistical time series, which consists of the study of factors that lead to changes in the dynamics of the studied process. This approach potentially has better indicators of adequacy, accuracy, and efficiency in obtaining final solutions than classical approaches. The implementation of this approach is shown using an example of the analysis of exchange rate changes. The obtained results show the practicality of considering multi-factoriality in forecasting tasks.

Overcoming the Challenges of Data Lack, Leakage, and Dimensionality in Intrusion Detection Systems: A Comprehensive Review

The Internet of Things (IoT) and cloud computing are rapidly gaining momentum as decentralized internet-based technologies and have led to an increase in information in nearly every technical and commercial industry. However, ensuring the security of IoT systems is a pressing issue due to the complexities involved in connected and shared environments. Networks are guarded by Intrusion Detection Systems (IDS) against various cyber threats such as malware, viruses, and unauthorized access. IDS have recently adopted Machine Learning (ML) and Deep Learning (DL) techniques to identify and classify security risks. However, the effective utilization of these technologies depends on the availability, quality, and characteristics of the data used to train models. Moreover, data lack, data leak, and dimensionality (DLLD) are common problems in data science and ML. This paper surveys existing research and suggests solutions for overcoming DLLD-related issues to improve IDS model.

Fast and Space-Efficient Parallel Algorithms for Influence Maximization

Influence Maximization (IM) is a crucial problem in data science. The goal is to find a fixed-size set of highly influential seed vertices on a network to maximize the influence spread along the edges. While IM is NP-hard on commonly used diffusion models, a greedy algorithm can achieve (1 - 1/ e )-approximation by repeatedly selecting the vertex with the highest marginal gain in influence as the seed. However, we observe two performance issues in the existing work that prevent them from scaling to today's large-scale graphs: space-inefficient memorization to estimate marginal gain, and time-inefficient seed selection process due to a lack of parallelism. This paper significantly improves the scalability of IM using two key techniques. The first is a sketch-compression technique for the independent cascading model on undirected graphs. It allows combining the simulation and sketching approaches to achieve a time-space tradeoff. The second technique includes new data structures for parallel seed selection. Using our new approaches, we implemented PaC-IM : Parallel and Compressed IM. We compare PaC-IM with state-of-the-art parallel IM systems on a 96-core machine with 1.5TB memory. PaC-IM can process the ClueWeb graph with 978M vertices and 75B edges in about 2 hours. On average, across all tested graphs, our uncompressed version is 5--18x faster and about 1.4x more space-efficient than existing parallel IM systems. Using compression further saves 3.8x space with only 70% overhead in time on average.

An external attention-based feature ranker for large-scale feature selection

An important problem in data science, feature selection (FS) consists of finding the optimal subset of features and eliminating irrelevant or redundant features. The FS task on high-dimensional data is challenging for the FS methods currently available in the literature. To overcome this limitation, we propose a novel feature selection method called External Attention-Based Feature Ranker for Large-Scale Feature Selection (EAR-FS) whose function is based on the logic of an attention mechanism and a hybrid metaheuristic. EAR-FS comprises three interdependent modules: (1) in the training module design, a multilayer perceptron network endowed with an attention module is trained to fit the dataset; (2) in feature ranking by attention, the trained attention module is used for attention updating and to rank features according to their importance; 3) in subset generation, a two-stage heuristic approach is applied to determine a small number of features that still guarantee high-accuracy performance. The experimental benchmark comprised 26 datasets of small, large and very large sizes, ranging from 15 to 12,533 features. Experiments performed against the state-of-the-art algorithms of FS show that our algorithm is efficient at selecting a small number of features from large datasets while guaranteeing excellent levels of classification accuracy. For instance, EAR-FS demonstrated its capability to reduce the features of the 11 Tumor dataset by 97% while maintaining a classifier accuracy of over 93%.

Two-Way Concept-Cognitive Learning via Concept Movement Viewpoint.

Representation and learning of concepts are critical problems in data science and cognitive science. However, the existing research about concept learning has one prevalent disadvantage: incomplete and complex cognitive. Meanwhile, as a practical mathematical tool for concept representation and concept learning, two-way learning (2WL) also has some issues leading to the stagnation of its related research: the concept can only learn from specific information granules and lacks a concept evolution mechanism. To overcome these challenges, we propose the two-way concept-cognitive learning (TCCL) method for enhancing the flexibility and evolution ability of 2WL for concept learning. We first analyze the fundamental relationship between two-way granule concepts in the cognitive system to build a novel cognitive mechanism. Furthermore, the movement three-way decision (M-3WD) method is introduced to 2WL to study the concept evolution mechanism via the concept movement viewpoint. Unlike the existing 2WL method, the primary consideration of TCCL is two-way concept evolution rather than information granules transformation. Finally, to interpret and help understand TCCL, an example analysis and some experiments on various datasets are carried out to demonstrate our method's effectiveness. The results show that TCCL is more flexible and less time-consuming than 2WL, and meanwhile, TCCL can also learn the same concept as the latter method in concept learning. In addition, from the perspective of concept learning ability, TCCL is more generalization of concepts than the granule concept cognitive learning model (CCLM).

Effect of Feature Selection on the Accuracy of Machine Learning Model

In real life data science problems, it’s almost rare that all the features in the dataset are useful for building a model. In machine learning, feature selection is the process of selecting a subset of relevant features or attributes for constructing a model. Removing irrelevant and redundant features and, selecting relevant features will improve the accuracy of a machine learning model. Furthermore, adding unnecessary variables to a model increases the overall complexity of the model. Our experiment indicates that the accuracy of a classification model is highly affected by the process of feature selection. We train three algorithms (K-Nearest Neighbors, Decision Tree, Multi-layer Perceptron) by selecting all the features and we got accuracies 49%, 84% and 71% accordingly. After doing some feature selection without any logical changes in models code the accuracy scores jumped to 82%, 86% and 78% accordingly which is quite impressive.

Elements of artificial intelligence in a predictive personalized model of pharmacotherapy choice in patients with heart failure with mildly reduced ejection fraction of ischemic origin

Aim. To create and train a neural network (NN) of a predictive personalized model of pharmacotherapy choice in patients with heart failure with mildly reduced ejection fraction (HFmrEF) of ischemic origin.Material and methods. The study included 170 people with HFmrEF of ischemic origin, who on the background standard pharmacotherapy, received a beta-blocker (BB) or BB+mineralocorticoid receptor antagonist eplerenone (EP): bisoprolol (BIS); BIS+EP; nebivolol (NEB); NEB+EP. Patients underwent echocardiography and were analyzed for serum aldosterone (AL), tumor necrosis factor-α (TNF-α), matrix metalloproteinase 9 (MMP-9). To create the NN model, the following approximate predictive function of parameters was used: age, AL, TNF-α, MMP-9, sphericity index (SI), type of pharmacotherapy. The result of this function is a parameter vector: AL, TNF-α, MMP-9, SI and quality of life (QOL). The designed NN model is implemented in the Matlab software package for solving machine learning and Data Science problems. The NN model is represented as a connected graph and NN function. Dichotomous analysis was used to compare the effect of treatment types in pairs. For intergroup comparison of therapy, the Wilcoxon W test method. The critical significance (p) was considered <0,05.Results. As a result of model inference, the predicted clinical parameters of patients were obtained, depending on the influence of pharmacotherapy type on the levels of AL, TNF-α, MMP-9, and SI. Function approximation of the distribution was constructed. Determination coefficient R2 of approximating functions was ≥0,92. The calculated values for the BIS therapy groups were obtained; BIS+EP — 169,59, 82,30, 15,26 and 52,92; NEB — 186,42, 87,65, 16,10 and 57,22; NEB+EP — 171,17, 71,90, 14,22 and 58,68, respectively. There were following mean values in the vector of initial states (before therapy): AL, MMP-9, TNF-α, and QOL — 205,84, 174,16, 18,32, and 50,71, respectively. The greatest negative changes of AL, MMP-9, TNF-α (p<0,05) was observed in the NEP+EP group.Conclusion. In the course of the study, using artificial intelligence, a predictive model of a personalized approach to pharmacotherapy choice in patients with HFmrEF of ischemic origin was developed and trained. It has been established that NEP+EP therapy has the greatest effect.

Neural Integro-Differential Equations

Modeling continuous dynamical systems from discretely sampled observations is a fundamental problem in data science. Often, such dynamics are the result of non-local processes that present an integral over time. As such, these systems are modeled with Integro-Differential Equations (IDEs); generalizations of differential equations that comprise both an integral and a differential component. For example, brain dynamics are not accurately modeled by differential equations since their behavior is non-Markovian, i.e. dynamics are in part dictated by history. Here, we introduce the Neural IDE (NIDE), a novel deep learning framework based on the theory of IDEs where integral operators are learned using neural networks. We test NIDE on several toy and brain activity datasets and demonstrate that NIDE outperforms other models. These tasks include time extrapolation as well as predicting dynamics from unseen initial conditions, which we test on whole-cortex activity recordings in freely behaving mice. Further, we show that NIDE can decompose dynamics into their Markovian and non-Markovian constituents, via the learned integral operator, which we test on fMRI brain activity recordings of people on ketamine. Finally, the integrand of the integral operator provides a latent space that gives insight into the underlying dynamics, which we demonstrate on wide-field brain imaging recordings. Altogether, NIDE is a novel approach that enables modeling of complex non-local dynamics with neural networks.

EXtreme Gradient Boosting Algorithm with Machine Learning: a Review

The primary task of machine learning is to extract valuable information from the data that is generated every day, process it to learn from it, and take useful actions. Original language process, pattern detection, search engines, medical diagnostics, bioinformatics, and chemical informatics are all examples of application areas for machine learning. XGBoost is a recently released machine learning algorithm that has shown exceptional capability for modeling complex systems and is the most superior machine learning algorithm in terms of prediction accuracy and interpretability and classification versatility. XGBoost is an enhanced distributed scaling enhancement library that is built to be extremely powerful, adaptable, and portable. It uses augmented scaling to incorporate machine learning algorithms. it is a parallel tree boost that addresses a variety of data science problems quickly and accurately. Python remains the language of choice for scientific computing, data science, and machine learning, which boosts performance and productivity by enabling the use of clean low-level libraries and high-level APIs. This paper presents one of the most prominent supervised and semi-supervised learning (SSL) machine learning algorithms in a Python environment.

Anomaly detection in the probability simplex under different geometries

AbstractAn open problem in data science is that of anomaly detection. Anomalies are instances that do not maintain a certain property that is present in the remaining observations in a dataset. Several anomaly detection algorithms exist, since the process itself is ill-posed mainly because the criteria that separates common or expected vectors from anomalies are not unique. In the most extreme case, data is not labelled and the algorithm has to identify the vectors that are anomalous, or assign a degree of anomaly to each vector. The majority of anomaly detection algorithms do not make any assumptions about the properties of the feature space in which observations are embedded, which may affect the results when those spaces present certain properties. For instance, compositional data such as normalized histograms, that can be embedded in a probability simplex, constitute a particularly relevant case. In this contribution, we address the problem of detecting anomalies in the probability simplex, relying on concepts from Information Geometry, mainly by focusing our efforts in the distance functions commonly applied in that context. We report the results of a series of experiments and conclude that when a specific distance-based anomaly detection algorithm relies on Information Geometry-related distance functions instead of the Euclidean distance, the performance is significantly improved.

Fast and provable tensor robust principal component analysis via scaled gradient descent

Abstract An increasing number of data science and machine learning problems rely on computation with tensors, which better capture the multi-way relationships and interactions of data than matrices. When tapping into this critical advantage, a key challenge is to develop computationally efficient and provably correct algorithms for extracting useful information from tensor data that are simultaneously robust to corruptions and ill-conditioning. This paper tackles tensor robust principal component analysis (RPCA), which aims to recover a low-rank tensor from its observations contaminated by sparse corruptions, under the Tucker decomposition. To minimize the computation and memory footprints, we propose to directly recover the low-dimensional tensor factors—starting from a tailored spectral initialization—via scaled gradient descent (ScaledGD), coupled with an iteration-varying thresholding operation to adaptively remove the impact of corruptions. Theoretically, we establish that the proposed algorithm converges linearly to the true low-rank tensor at a constant rate that is independent with its condition number, as long as the level of corruptions is not too large. Empirically, we demonstrate that the proposed algorithm achieves better and more scalable performance than state-of-the-art tensor RPCA algorithms through synthetic experiments and real-world applications.

Tug of War games and PDEs on graphs with applications in image and high dimensional data processing

The aim of this note is to revisit the connections between some stochastic games, namely Tug-of-War games, and a class of nonlocal PDEs on graphs. We consider a general formulation of Tug-of-War games which is shown to be related to many classical PDEs in the continuous setting. We transcribe these equations on graphs using ad hoc differential operators and we show that it covers several nonlocal PDEs on graphs such as infty -Laplacian, game p-Laplacian and the eikonal equation. This unifying mathematical framework allows us to easily design simple algorithms to solve several inverse problems in imaging and data science, with a particular focus on cultural heritage and medical imaging.

Kernel Approximation on Algebraic Varieties

Low-rank approximation of kernels is a fundamental mathematical problem with widespread algorithmic applications. Often the kernel is restricted to an algebraic variety, e.g., in problems involving sparse or low-rank data. We show that significantly better approximations are obtainable in this setting: the rank required to achieve a given error depends on the variety’s dimension rather than the ambient dimension, which is typically much larger. This is true in both high-precision and high-dimensional regimes. Our results are presented for smooth isotropic kernels, the predominant class of kernels used in applications. Our main technical insight is to approximate smooth kernels by polynomial kernels and to leverage two key properties of polynomial kernels that hold when they are restricted to a variety. First, their ranks decrease exponentially in the variety’s co-dimension. Second, their maximum values are governed by their values over a small set of points. Together, our results provide a general approach for exploiting (approximate) “algebraic structure” in datasets in order to efficiently solve large-scale data science problems.

Identifying bird species by their calls in Soundscapes

In many real data science problems, it is common to encounter a domain mismatch between the training and testing datasets, which means that solutions designed for one may not transfer well to the other due to their differences. An example of such was in the BirdCLEF2021 Kaggle competition, where participants had to identify all bird species that could be heard in audio recordings. Thus, multi-label classifiers, capable of coping with domain mismatch, were required. In addition, classifiers needed to be resilient to a long-tailed (imbalanced) class distribution and weak labels. Throughout the competition, a diverse range of solutions based on convolutional neural networks were proposed. However, it is unclear how different solution components contribute to overall performance. In this work, we contextualise the problem with respect to the previously existing literature, analysing and discussing the choices made by the different participants. We also propose a modular solution architecture to empirically quantify the effects of different architectures. The results of this study provide insights into which components worked well for this challenge.

Can neural networks extrapolate? Discussion of a theorem by Pedro Domingos

Neural networks trained on large datasets by minimizing a loss have become the state-of-the-art approach for resolving data science problems, particularly in computer vision, image processing and natural language processing. In spite of their striking results, our theoretical understanding about how neural networks operate is limited. In particular, what are the extrapolation capabilities of trained neural networks if any? In this paper we discuss a theorem of Domingos stating that “every machine learned by continuous gradient descent is approximately a kernel machine”. According to Domingos, this fact leads to conclude that all machines trained on data are mere kernel machines. We first extend Domingo’s result in the discrete case and to networks with vector-valued output. We then study its relevance and significance on simple examples. We find that in simple cases, the “neural tangent kernel” arising in Domingos’ theorem does provide understanding of the networks’ predictions. When the task given to the network grows in complexity, the interpolation capability of the network can be effectively explained by Domingos’ theorem, and no extrapolation capability of the network beyond its learning domain is found, even when the network’s structure would allow for it. We illustrate this fact on a classic perception theory problem: recovering a shape from its boundary.

Editorial: Tensor numerical methods and their application in scientific computing and data science

Editorial: Tensor numerical methods and their application in scientific computing and data science

Analysis of Kernel Matrices via the von Neumann Entropy and Its Relation to RVM Performances.

Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model's fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited.