High-dimensional Data Research Articles

A two-stage monitoring scheme for multiple high-dimensional data streams

With the rapid development of data acquisition and analysis technology, the online monitoring of data streams is particularly important in modern industries. This study proposes an efficient online monitoring scheme for multiple high-dimensional data streams. We first construct a point-wise process monitoring control chart based on statistical extreme value theory. Then we use the clustering method to roughly obtain reliable data streams and suspected abnormal data streams, within the sliding window range, further identify abnormal data streams one by one over time points, and estimate the potential change points of these abnormal streams. Finally, we conduct simulations and case studies on this method, and compare it with existing competitive methods to demonstrate its effectiveness.

From Text to Recommendations: How Vector Databases are Revolutionizing Personalized Content Delivery

Abstract: Effective similarity search and retrieval are now possible thanks to vector databases, which have become a potent tool for organizing and searching high-dimensional data. This article provides a thorough examination of vector databases, their underlying theories, and their applications across a range of industries. In handling complex data types, we address the significance of vector representation and emphasize the benefits of vector databases over traditional databases [1]. The article explores the process of creating a vector database, highlighting the critical function of indexing strategies such as IVF (Inverted File Indexing) and HNSW (Hierarchical Navigable Small World) in guaranteeing the effectiveness and precision of searches [2]. In addition, we discuss the problems caused by the curse of dimensionality and offer solutions to lessen its effects on nearest neighbor searches [3]

Asymptotic behavior of some multicategory classification methods for high-dimensional data

We consider multicategory extensions of binary discrimination methods via one-versus-one (OVO) or one-versus-rest (OVR) methodologies, focusing on extensions of the binary classification by linear mean difference (MD), support vector machine (SVM), maximal data piling (MDP), and distance weighted discrimination (DWD) via OVO, and the multicategory extension of MD via OVR, in the context of high-dimensional and low sample size (HDLSS) data. The asymptotic behavior of OVO-MD, OVO-SVM, OVO-MDP and OVO-DWD is described when the dimension of the data increases and the sample size is fixed, in terms of the probabilities of correct classification of a new data point, finding sufficient conditions for the correct classification probabilities to converge to one as the dimension approaches infinity. As in the binary case, OVO-MD, OVO-SVM and OVO-MDP have the same asymptotic behavior while OVO-DWD could behave differently. We also consider the asymptotic behavior of the OVR-MD methodology providing necessary and sufficient conditions for a new data point of a given class to be correctly classified with probability tending to one. A simulation experiment is conducted to further compare the methodologies, and consider the four binary methods in the OVR case. We evaluate the performance of the considered methods using a microarray data set.

GBDT Method Integrating Feature-Enhancement and Active-Learning Strategies-Sea Ice Thickness Inversion in Beaufort Sea.

Sea ice, as an important component of the Earth's ecosystem, has a profound impact on global climate and human activities due to its thickness. Therefore, the inversion of sea ice thickness has important research significance. Due to environmental and equipment-related limitations, the number of samples available for remote sensing inversion is currently insufficient. At high spatial resolutions, remote sensing data contain limited information and noise interference, which seriously affect the accuracy of sea ice thickness inversion. In response to the above issues, we conducted experiments using ice draft data from the Beaufort Sea and designed an improved GBDT method that integrates feature-enhancement and active-learning strategies (IFEAL-GBDT). In this method, the incident angle and time series are used to perform spatiotemporal correction of the data, reducing both temporal and spatial impacts. Meanwhile, based on the original polarization information, effective multi-attribute features are generated to expand the information content and improve the separability of sea ice with different thicknesses. Taking into account the growth cycle and age of sea ice, attributes were added for month and seawater temperature. In addition, we studied an active learning strategy based on the maximum standard deviation to select more informative and representative samples and improve the model's generalization ability. The improved GBDT model was used for training and prediction, offering advantages in dealing with nonlinear, high-dimensional data, and data noise problems, further expanding the effectiveness of feature-enhancement and active-learning strategies. Compared with other methods, the method proposed in this paper achieves the best inversion accuracy, with an average absolute error of 8 cm and a root mean square error of 13.7 cm for IFEAL-GBDT and a correlation coefficient of 0.912. This research proves the effectiveness of our method, which is suitable for the high-precision inversion of sea ice thickness determined using Sentinel-1 data.

The Application of K-means Clustering Algorithm in the Evaluation of E-Commerce Websites

When dealing with large amounts of high-dimensional transaction data, clustering approaches often struggle with challenges including elasticity, weak processing capabilities of high-dimensional data, sensitivity to data sequence across time, independence of parameters, and ability to manage noise. These problems may prevent the methods from providing accurate predictions. Experiments conducted with data samples collected from 300 different mobile phones purchased on Taobao yielded the following results. K-means beats Single-pass in evaluating e-commerce transactions because of its higher intra-class dissimilarity and inter-class similarity. K-means clustering is an approach to the organization of massive datasets that is both effective and flexible. The outcome of the clustering algorithm is sensitive not only to the total number of clusters but also to how they were initially arranged. Because of this, it is simple to demonstrate that the results of clustering are best optimized locally. For this reason, continuing research into the elements that influence the number of clusters produced by this method as well as the starting locations of the clustering center is a crucial endeavor.

The Application and Exploration of Data Visualization Technology in Artistic Expression and Exhibition

Embarking on the transformation of virtual art museums, the confluence of data visualization technology and deep learning emerges as a catalyst for redefining artistic expression. Driven by the desire to change traditional approaches, this work utilizes a sophisticated deep learning model, the self-attention-based cycle-consistent generative adversarial network (SA-CCGAN), for transformative purposes. The objective of proposed Modernizing Virtual Art: The Collaboration of Data Visualization Technology and Self-Attention-Based Cycle-Consistent Generative Adversarial Network for Artistic Expression (MVA: DVT & SA-CCGAN) is to elevate artistic expression and exhibition practices within the virtual space, marking a paradigm shift in traditional approaches. SA-CCGAN, renowned for its ability to generate realistic and coherent artistic representations, serves as the cornerstone of this transformative endeavor. The workflow intricately incorporates SA-CCGAN into the virtual art museum context, enhancing stylized outcomes and capturing global geometric features. This yields a computationally efficient representation of high-dimensional visual data, revolutionizing digital art presentation. The fusion of data visualization technology, creativity, and meticulous design not only signals innovation but also has the potential to redefine the virtual art landscape, fostering accessibility, engagement, and a deeper cultural exchange in digital art exploration.

Benchmarking Unsupervised Clustering Algorithms for Atomic Force Microscopy Data on Polyhydroxyalkanoate Films.

Surface of polyhydroxyalkanoate (PHA) films of varying monomer compositions are analyzed using atomic force microscopy (AFM) and unsupervised machine learning (ML) algorithms to investigate and classify films based on global attributes such as the scan size, film thickness, and monomer type. The experiment provides benchmarked results for 12 of the most widely used clustering algorithms via a hybrid investigation approach while highlighting the impact of using the Fourier transform (FT) on high-dimensional vectorized data for classification on various pools of data. Our findings indicate that the use of a one-dimensional (1D) FT of vectorized data produces the most accurate outcome. The experiment also provides insights into case-by-case investigations of algorithm performances and the impact of various data pools. Lastly, we show an early version of our tool aimed at investigating surfaces using ML approaches and discuss the results of our current experiment to configure future improvements.

Construction of time series causal network based on partial rank correlation

Discovering causal relationships from time series data is a significant research area in data mining. To tackle the challenges of identifying and constructing causal networks in non-linear, high-dimensional time series data, we introduce the partial rank correlation coefficient and propose two new structure learning algorithms suitable for time-series causal network modeling. In this paper, we make three main contributions. Firstly, we demonstrate the suitability of the partial rank correlation as a criterion for independence testing. Secondly, by integrating the partial rank correlation with constraint-based causal discovery methods, we propose the TS-PRCS algorithm for temporal causal network discovery. Thirdly, we combine ideas from local learning, constraint-based methods, and score-based search to propose the TS-MCHC algorithm, which combines mixed constraints and hill-climbing search for time-series causal discovery. Finally, we validate the effectiveness of the TS-PRCS and TS-MCHC algorithms through experiments on both simulated and real-world data. Compared to existing methods, our proposed algorithms exhibit robust causal detection capabilities in diverse non-linear environments and demonstrate efficient performance in high-dimensional time series networks. The experimental results affirm the ability of our methods to effectively uncover causal relationships among time series data, presenting promising prospects for practical applications.

Distributed Denial of Service Attack Detection in IoT Networks using Deep Learning and Feature Fusion: A Review

The explosive growth of Internet of Things (IoT) devices has led to escalating threats from distributed denial of service (DDoS) attacks. Moreover, the scale and heterogeneity of IoT environments pose unique security challenges, and intelligent solutions tailored for the IoT are needed to defend critical infrastructure. The deep learning technique shows great promise because automatic feature learning capabilities are well suited for the complex and high-dimensional data of IoT systems. Additionally, feature fusion approaches have gained traction in enhancing the performance of deep learning models by combining complementary feature sets extracted from multiple data sources. This paper aims to provide a comprehensive literature review focused specifically on deep learning techniques and feature fusion for DDoS attack detection in IoT networks. Studies employing diverse deep learning models and feature fusion techniques are analysed, highlighting key trends and developments in this crucial domain. This review provides several significant contributions, including an overview of various types of DDoS attacks, a comparison of existing surveys, and a thorough examination of recent applications of deep learning and feature fusion for detecting DDoS attacks in IoT networks. Importantly, it highlights the current challenges and limitations of these deep learning techniques based on the literature surveyed. This review concludes by suggesting promising areas for further research to enhance deep learning security solutions, which are specifically tailored to safeguarding the fast-growing IoT infrastructure against DDoS attacks.

A multi-energy load forecasting method based on complementary ensemble empirical model decomposition and composite evaluation factor reconstruction

Ensuring precise multi-energy load forecasting is crucial for the effective planning, management, and operation of Integrated Energy Systems (IES). This study proposes a novel multivariate load forecasting model based on time-series decomposition and reconstruction to handle the complex, high-dimensional multi-energy load data in IES and enhance forecasting accuracy. Initially, the model conducts a thorough correlation analysis and variable screening to minimize irrelevant data interference. It then applies denoising by decomposing the load sequence into modal components with distinct characteristics, using the complementary ensemble empirical mode decomposition (CEEMD). To overcome the unstable prediction accuracy inherent in time-domain decomposition methods, this study introduces an innovative composite evaluation factor (CEF) that reconstructs the modal components after considering their complexity, coupling, and frequency. The final predictions are generated using the proposed MTL-CNN-BiLSTM model, optimized with the attention mechanism. The results show that the proposed model significantly reduces error accumulation compared to traditional time-domain analysis methods, achieving a 37.40% reduction in average forecasting error and a 30.73% increase in forecasting efficiency.

Application of Deep Learning for the Detection of Genetic Variations: Its Implementation in Classifying Alzheimer's Disease

Deep learning emerges as a promising technique, utilizing nonlinear transformations for feature extraction from high-dimensional datasets. However, its application encounters challenges in genome-wide association studies (GWAS) dealing with high-dimensional genomic data. This study introduces an innovative three-step method termed SWAT-CNN for the identification of genetic variants. This approach employs deep learning to pinpoint phenotype-related single nucleotide polymorphisms (SNPs), facilitating the development of precise disease classification models. In the first step, the entire genome undergoes division into non overlapping fragments of an optimal size. Subsequently, convolutional neural network (CNN) analysis is conducted on each fragment to identify phenotype-associated segments. The second step, employs a Sliding Window Association Test (SWAT), where CNN is utilized on the selected fragments to compute phenotype influence scores (PIS) and detect phenotype-associated SNPs based on these scores. The third step involves running CNN on all identified SNPs to construct a comprehensive classification model. Validation of the proposed approach utilized GWAS data from the Alzheimer’s disease Neuroimaging Initiative (ADNI), encompassing 981 subjects, including cognitively normal older adults (CN) and individuals with Alzheimer's disease (AD). Notably, the method successfully identified the widely recognized APOE region as the most significant genetic locus for AD. The resulting classification model exhibited an area under the curve (AUC) of 0.82, demonstrating compatibility with traditional machine learning approaches such as random forest and XGBoost. SWAT-CNN, as a groundbreaking deep learning-based genome-wide methodology, not only identified AD-associated SNPs but also presented a robust classification model for Alzheimer's disease, suggesting potential applications across diverse biomedical domains.

A principal-weighted penalized regression model and its application in economic modeling

This paper introduces a novel Principal-Weighted Penalized (PWP) regression model, designed for dimensionality reduction in large datasets without sacrificing essential information. This new model retains the favorable features of the principal component analysis (PCA) technique and penalized regression models. It weighs the variables in a large data set based on their contributions to principal components identified by PCA, enhancing its capacity to uncover crucial hidden variables. The PWP model also efficiently performs variable selection and estimates regression coefficients through regularization. An application of the proposed model on high-dimensional economic data is studied. The results of comparative studies in simulations and a real example in economic modeling demonstrate its superior fitting and predictive abilities. The resulting model excels in accuracy and interpretability, outperforming existing methods.

An overview on deep clustering

In recent years, with the great success of deep learning and especially deep unsupervised learning, many deep architectural clustering methods, collectively known as deep clustering, have emerged. Deep clustering shows the potential to outperform traditional methods, especially in handling complex high-dimensional data, taking full advantage of deep learning. To achieve a comprehensive overview of the field of deep clustering, this review systematically explores deep clustering methods and their various applications. First, the basic network architecture of deep clustering is described in detail, including the common network frameworks, and loss functions. Subsequently, deep clustering is divided into several categories based on the network architecture, and benchmark datasets and evaluation metrics in the field are introduced. Next, the real-world applications of deep clustering are explored in depth, providing successful cases in the fields of bioinformatics, medicine, anomaly detection, and image processing, highlighting the broad applicability of deep clustering in solving real-world challenges. Finally, the paper summarizes its contributions and explores potential directions for future research in deep clustering.

Unveiling the Power of Data: A Journey Through Machine Learning Techniques

Machine Learning Techniques: Unveiling the Power of Data-Driven Solutions Machine learning (ML) has become a transformative force across disciplines, revolutionizing how computers extract knowledge from data and tackle complex tasks without explicit programming. This research paper delves into the core principles of machine learning, exploring its fundamental techniques and their far-reaching applications. We embark on a journey through the two main paradigms that govern learning: supervised and unsupervised. Supervised learning equips algorithms to make informed predictions based on labeled data, encompassing tasks like spam filtering, disease diagnosis, and stock market forecasting. The algorithm ingests this data, which includes both the input features and the corresponding desired output, and learns the intricate relationships between them. This empowers it to generalize this knowledge to unseen data, enabling accurate predictions. Unsupervised learning, in contrast, thrives on unlabeled data, where the data points lack predefined categories or target variables. Here, the objective is to uncover the inherent structures or patterns within the data itself. Unsupervised learning techniques are invaluable for tasks like data clustering, where similar data points are grouped together to reveal underlying relationships. Additionally, they play a crucial role in dimensionality reduction, a process of compressing high-dimensional data into a lower-dimensional space for efficient processing and analysis. This is particularly beneficial when dealing with massive datasets that may overwhelm traditional computational methods. Beyond these fundamental paradigms, the landscape of machine learning extends further. Deep learning, a subfield inspired by the structure and function of the human brain, utilizes artificial neural networks with multiple layers. These complex networks are adept at learning intricate patterns from vast amounts of data, achieving remarkable success in areas like computer vision, natural language processing, and speech recognition. Deep learning models, for instance, power facial recognition technology, enabling our devices to identify individuals with exceptional accuracy. In the realm of natural language processing, deep learning algorithms fuel machine translation systems, allowing seamless communication across languages. Reinforcement learning presents another powerful approach. Here, an agent interacts with an environment, learning through trial and error. The agent receives rewards for desired actions and penalties for undesired ones, constantly refining its decision-making process. This approach has applications in robotics, where robots can learn to navigate complex environments and perform tasks with increasing autonomy. Reinforcement learning is also used in game playing, where AI agents can learn optimal strategies through self-play and achieve superhuman levels of performance. As machine learning continues its exponential growth, ethical considerations become paramount. Potential biases within the training data can lead to discriminatory outcomes in areas like loan approvals or facial recognition software. Additionally, the explainability and transparency of complex models, particularly deep learning models, raise concerns. To ensure responsible development and deployment of ML, addressing these issues is crucial. The field requires robust methodologies to mitigate bias in training data and develop techniques that make complex models interpretable, allowing us to understand the rationale behind their predictions. Looking towards the horizon, the future of machine learning is brimming with potential. From personalized healthcare with tailored treatment plans to self-driving cars that navigate roads with exceptional precision, ML promises to revolutionize various aspects of our lives. Continuous research and development will enhance the capabilities and robustness of machine learning models, further expanding their reach and impact. By harnessing the power of data and fostering responsible development practices, machine learning will undoubtedly play a pivotal role in shaping a more intelligent and data-driven future

A model-free variable screening method for optimal treatment regimes with high-dimensional survival data

Summary We propose a model-free variable screening method for the optimal treatment regime with high-dimensional survival data. The proposed screening method provides a unified framework to select the active variables in a prespecified target population, including the treated group as a special case. Based on this framework, the optimal treatment regime is exactly the optimal classifier that minimizes a weighted misclassification error rate, with weights associated with survival outcome variables, the censoring distribution, and a prespecified target population. Our main contribution involves reformulating the weighted classification problem into a classification problem within a hypothetical population, where the observed data can be viewed as a sample obtained from outcome-dependent sampling, with the selection probability inversely proportional to the weights. Consequently, we introduce the weighted Kolmogorov–Smirnov approach for selecting active variables in the optimal treatment regime, extending the conventional Kolmogorov–Smirnov method for binary classification. Additionally, the proposed screening method exhibits two levels of robustness. The first level of robustness is achieved because the proposed method does not require any model assumptions for survival outcome on treatment and covariates, whereas the other is attained as the form of treatment regimes is allowed to be unspecified even without requiring convex surrogate loss, such as logit loss or hinge loss. As a result, the proposed screening method is robust to model misspecifications, and nonparametric learning methods such as random forests and boosting can be applied to those selected variables for further analysis. The theoretical properties of the proposed method are established. The performance of the proposed method is examined through simulation studies and illustrated by a real dataset.

Development and Implementation of a Multilayer Deep Learning-Based Bank Credit Risk Forecasting System

The complexity of the financial environment and the international community makes the capital flow face various challenges, and it is difficult to obtain accurate credit prediction results in the actual application environment. Considering the complex non-linear characteristics of customer information, the Analytic Hierarchy Process is studied to meet the needs of bank credit risk assessment. On this basis, a depth neural network with different complexities was selected for the three indicators built to classify the features. The composition of the neural network module and the number of neurons were determined by experiment, and Dropout was used to prevent overfitting of the test dataset. Stability and ablation experiments showed that the model can control the error between datasets to 0.021. The ablation experiment showed that the numbers of hidden layers and neurons were the best. Simulation tests showed that the sensitivity and accuracy of this method were 85.25% and 92.55%, respectively, which were superior to other classification methods. The real data of banks in the past four years were tested. The results could accurately classify the risks of enterprises and individual customers, and the results of stress test showed that the model is stable. It is found that traditional credit risk assessment models rely on statistical means and rule decisions, and these methods may not fully reveal the complex non-linear relationship and the internal relationship of financial indicators in high-dimensional data. The combination of deep learning technology and hierarchical analysis can better deal with and explain the complex non-linear problems in bank risk assessment.

Machine Learning and Feature Selection for soil spectroscopy. An evaluation of Random Forest wrappers to predict soil organic matter, clay, and carbonates

Soil spectroscopy estimates soil properties using the absorption features in soil spectra. However, modelling soil properties with soil spectroscopy is challenging due to the high dimensionality of spectral data. Feature Selection wrapper methods are promising approaches to reduce the dimensionality but are barely used in soil spectroscopy. The aim of this study is to evaluate the performance of two feature selection wrapper methods, Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS) built using the Random Forest (RF) algorithm, for dimensionality reduction of spectral data and predictive modelling of modelling soil organic matter (SOM), clay and carbonates. The reflectance of 100 soil samples, acquired from Sierra de las Nieves (Spain), was measured under laboratory conditions using ASD FieldSpec Pro JR. Four different datasets were obtained after applying two spectral preprocessing methods to raw spectra: raw spectra, Continuum Removal (CR), Multiplicative Scatter Correction (MSC), and a so-called “Global” dataset composed of raw, CR and MSC features. The performance of RF models built with feature selection methods was compared to that of Partial Least Squares Regression (PLSR) and RF (alone).RF models built with SFS and SFFS outperformed PLSR and RF alone models: The best RF models with feature selection had a respective ratio of performance to interquartile distance of 1.93, 0.38 and 2.56. PLSR models had an accuracy of 1.41, 0.29 and 1.81 for SOM, carbonates, and clay, respectively. RF alone had a respective performance of 1.29, 0.29 and 1.81. The application of feature selection wrapper methods reduced the number of features to less than 1 % of the starting features. Features were selected across all spectra for SOM and clay, and around 900 nm, 1900 nm, and 2350 nm for carbonates. However, feature selection highlighted features around 1100 nm in SOM modelling, as well as other features around 2200 nm, which is considered a main absorption feature of clay. The application of feature selection with Random Forest was very important in improving modelling accuracy, reducing the redundant features and avoiding the curse of dimensionality or Hughes effect. Thus, this research showed an alternative to dimensionality reduction approaches that have been applied to date to model soil properties with spectroscopy and paves the way for further scientific investigation based on feature selection methods and machine learning.

Variational mode decomposition–based nonstationary coherent structure analysis for spatiotemporal data

The conventional modal analysis techniques face difficulties in handling nonstationary phenomena, such as transient, nonperiodic, or intermittent phenomena. This paper presents a variational mode decomposition–based nonstationary coherent structure (VMD-NCS) analysis that enables the extraction and analysis of coherent structures in the case of nonstationary phenomena from high-dimensional spatiotemporal data. The VMD-NCS analysis decomposes the input spatiotemporal data into intrinsic coherent structures (ICSs) that represent nonstationary spatiotemporal patterns and exhibit coherence in both spatial and temporal directions. Furthermore, unlike many conventional modal analysis techniques, the proposed method accounts for the temporal changes in the spatial distribution with time. The performance of the VMD-NCS analysis was validated based on the transient growth phenomena in the flow around a cylinder. It was confirmed that the temporal changes in the spatial distribution, depicting the transient growth of vortex shedding where fluctuations arising in the far-wake region gradually approach the near-wake region, were represented as a single ICS. Furthermore, in the analysis of the quasi-periodic flow field around a pitching airfoil, the temporal changes in the spatial distribution and the amplitude of vortex shedding behind the airfoil, influenced by the pitching motion of the airfoil, were captured as a single ICS. Additionally, the impact of two parameters that control the number of ICSs (K) and the penalty factor related to the temporal coherence (α), was investigated. The results revealed that K has a significant impact on the VMD-NCS analysis results. In the case of a relatively high K, the VMD-NCS analysis tends to extract more periodic spatiotemporal patterns resembling the results of dynamic mode decomposition. In the case of a small K, it tends to extract more nonstationary spatiotemporal patterns.

CARE: Large Precision Matrix Estimation for Compositional Data

High-dimensional compositional data are prevalent in many applications. The simplex constraint poses intrinsic challenges to inferring the conditional dependence relationships among the components forming a composition, as encoded by a large precision matrix. We introduce a precise specification of the compositional precision matrix and relate it to its basis counterpart, which is shown to be asymptotically identifiable under suitable sparsity assumptions. By exploiting this connection, we propose a composition adaptive regularized estimation (CARE) method for estimating the sparse basis precision matrix. We derive rates of convergence for the estimator and provide theoretical guarantees on support recovery and data-driven parameter tuning. Our theory reveals an intriguing tradeoff between identification and estimation, thereby highlighting the blessing of dimensionality in compositional data analysis. In particular, in sufficiently high dimensions, the CARE estimator achieves minimax optimality and performs as well as if the basis were observed. We further discuss how our framework can be extended to handle data containing zeros, including sampling zeros and structural zeros. The advantages of CARE over existing methods are illustrated by simulation studies and an application to inferring microbial ecological networks in the human gut. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A novel intuitionistic fuzzy rough instance selection and attribute reduction with kernelized intuitionistic fuzzy C-means clustering to handle imbalanced datasets

Due to advancement of internet and lab based technologies, large volume of high dimensional data are generated every day. These data usually consisted of several issues such as class imbalance, noise, later uncertainty, irrelevant and/or redundant features, and redundancy in size. These issues degrade the overall performance measures of the various machine learning algorithms. An efficient method to cope with such issues for a large sized datasets is to apply efficient data reduction techniques. In the recent years, numerous data reduction techniques have been presented based on fuzzy rough set theory to tackle these obstacles. However, building such a method that can handle all the above mentioned issues simultaneously is still a challenging task. In this study, we handle all these obstacles simultaneously by introducing a novel approach to obtain the reduced dataset by combining kernelized intuitionistic fuzzy C-means with intuitionistic fuzzy rough set model. Intuitionistic fuzzy rough set handles the vagueness and uncertainty in a better way than fuzzy set aided models as it takes use of membership, non-membership as well as hesitancy to capture the uncertainty of real-valued datasets. To generate the membership and non-membership grades, kernelized intuitionistic fuzzy C- means based notion is introduced. Further, an intuitionistic fuzzy rough set model is established by addressing lower and upper approximations based on a novel similarity relation. Moreover, all the necessary conditions are justified with the relevant mathematical theorems. Next, this model is employed for dimensionality reduction based on the concept of discernibility matrix by using the idea of different classes’ ratios to avoid the noise. Thereafter, positive region is defined with the help of lower approximation. The positive region information is applied to tackle problematic instances available in both minority and majority class for the imbalanced dataset after generation of artificial samples by synthetic minority oversampling technique (SMOTE). A comprehensive experimental study is added to show the effectiveness of the proposed technique. Finally, a framework is established to improve the prediction of animal toxin peptides.