Dimensionality Reduction Of Datasets Research Articles

Precision cancer classification using liquid biopsy and advanced machine learning techniques

Cancer presents a significant global health burden, resulting in millions of annual deaths. Timely detection is critical for improving survival rates, offering a crucial window for timely medical interventions. Liquid biopsy, analyzing genetic variations, and mutations in circulating cell-free, circulating tumor DNA (cfDNA/ctDNA) or molecular biomarkers, has emerged as a tool for early detection. This study focuses on cancer detection using mutations in plasma cfDNA/ctDNA and protein biomarker concentrations. The proposed system initially calculates the correlation coefficient to identify correlated features, while mutual information assesses each feature's relevance to the target variable, eliminating redundant features to improve efficiency. The eXtrem Gradient Boosting (XGBoost) feature importance method iteratively selects the top ten features, resulting in a 60% dataset dimensionality reduction. The Light Gradient Boosting Machine (LGBM) model is employed for classification, optimizing its performance through a random search for hyper-parameters. Final predictions are obtained by ensembling LGBM models from tenfold cross-validation, weighted by their respective balanced accuracy, and averaged to get final predictions. Applying this methodology, the proposed system achieves 99.45% accuracy and 99.95% AUC for detecting the presence of cancer while achieving 93.94% accuracy and 97.81% AUC for cancer-type classification. Our methodology leads to enhanced healthcare outcomes for cancer patients.

Feature selection by multi-objective optimisation: Application to network anomaly detection by hierarchical self-organising maps

Feature selection is an important and active issue in clustering and classification problems. By choosing an adequate feature subset, a dataset dimensionality reduction is allowed, thus contributing to decreasing the classification computational complexity, and to improving the classifier performance by avoiding redundant or irrelevant features. Although feature selection can be formally defined as an optimisation problem with only one objective, that is, the classification accuracy obtained by using the selected feature subset, in recent years, some multi-objective approaches to this problem have been proposed. These either select features that not only improve the classification accuracy, but also the generalisation capability in case of supervised classifiers, or counterbalance the bias toward lower or higher numbers of features that present some methods used to validate the clustering/classification in case of unsupervised classifiers.The main contribution of this paper is a multi-objective approach for feature selection and its application to an unsupervised clustering procedure based on Growing Hierarchical Self-Organising Maps (GHSOMs) that includes a new method for unit labelling and efficient determination of the winning unit. In the network anomaly detection problem here considered, this multi-objective approach makes it possible not only to differentiate between normal and anomalous traffic but also among different anomalies. The efficiency of our proposals has been evaluated by using the well-known DARPA/NSL-KDD datasets that contain extracted features and labelled attacks from around 2million connections. The selected feature sets computed in our experiments provide detection rates up to 99.8% with normal traffic and up to 99.6% with anomalous traffic, as well as accuracy values up to 99.12%.

Non-linear complex principal component analysis of nearshore bathymetry

Abstract. Complex principal component analysis (CPCA) is a useful linear method for dimensionality reduction of data sets characterized by propagating patterns, where the CPCA modes are linear functions of the complex principal component (CPC), consisting of an amplitude and a phase. The use of non-linear methods, such as the neural-network based circular non-linear principal component analysis (NLPCA.cir) and the recently developed non-linear complex principal component analysis (NLCPCA), may provide a more accurate description of data in case the lower-dimensional structure is non-linear. NLPCA.cir extracts non-linear phase information without amplitude variability, while NLCPCA is capable of extracting both. NLCPCA can thus be viewed as a non-linear generalization of CPCA. In this article, NLCPCA is applied to bathymetry data from the sandy barred beaches at Egmond aan Zee (Netherlands), the Hasaki coast (Japan) and Duck (North Carolina, USA) to examine how effective this new method is in comparison to CPCA and NLPCA.cir in representing propagating phenomena. At Duck, the underlying low-dimensional data structure is found to have linear phase and amplitude variability only and, accordingly, CPCA performs as well as NLCPCA. At Egmond, the reduced data structure contains non-linear spatial patterns (asymmetric bar/trough shapes) without much temporal amplitude variability and, consequently, is about equally well modelled by NLCPCA and NLPCA.cir. Finally, at Hasaki, the data structure displays not only non-linear spatial variability but also considerably temporal amplitude variability, and NLCPCA outperforms both CPCA and NLPCA.cir. Because it is difficult to know the structure of data in advance as to which one of the three models should be used, the generalized NLCPCA model can be used in each situation.