Dimensionality Reduction Techniques Research Articles

Towards an Improved High-Throughput Phenotyping Approach: Utilizing MLRA and Dimensionality Reduction Techniques for Transferring Hyperspectral Proximal-Based Model to Airborne Images

At present, it is critical to accurately monitor wheat crops to help decision-making processes in precision agriculture. This research aims to retrieve various wheat crop traits from hyperspectral data using machine learning regression algorithms (MLRAs) and dimensionality reduction (DR) techniques. This experiment was conducted in an agricultural field in Arborea, Oristano-Sardinia, Italy, with different factors such as cultivars, N-treatments, and soil ploughing conditions. Hyperspectral data were acquired on the ground using a full-range Spectral Evolution spectrometer (350–2500 nm). Four DR techniques, including (i) variable influence on projection (VIP), (ii) principal component analysis (PCA), (iii) vegetation indices (VIs), and (iv) spectroscopic feature (SF) calculation, were undertaken to reduce the dimension of the hyperspectral data while maintaining the information content. We used five MLRA models, including (i) partial least squares regression (PLSR), (ii) random forest (RF), (iii) support vector regression (SVR), (iv) Gaussian process regression (GPR), and (v) neural network (NN), to retrieve wheat traits at either leaf and canopy levels. The studied traits were leaf area index (LAI), leaf and canopy water content (LWC and CWC), leaf and canopy chlorophyll content (LCC and CCC), and leaf and canopy nitrogen content (LNC and CNC). MLRA models were able to accurately retrieve wheat traits at the canopy level with PLSR and NN indicating the highest modelling performance. On the contrary, MLRA models indicated less accurate retrievals of the leaf-level traits. DR techniques were found to notably improve the retrieval accuracy of crop traits. Furthermore, the generated models were re-calibrated using soil spectra and then transferred to an airborne dataset collected using a CASI-SASI hyperspectral sensor, allowing the estimation of wheat traits across the entire field. The predicted crop trait maps illustrated consistent patterns while also preserving the real-field characteristics well. Lastly, a statistical paired t-test was undertaken to conduct a proof of concept of wheat phenotyping analysis considering the different agricultural variables across the study site. N-treatment caused significant differences in wheat crop traits in many instances, whereas the observed differences were less pronounced between the cultivars. No particular impact of soil ploughing conditions on wheat crop characteristics was found. Using such combinations of MLRA and DR techniques based on hyperspectral data can help to effectively monitor crop traits throughout the cropping seasons and can also be readily applied to other agricultural settings to help both precision farming applications and the implementation of high-throughput phenotyping solutions.

Fuzzy Evolutionary Algorithm in Construction Project for Valuation

Clustering algorithms has emerged as an important data mining technique for pattern recognition, data analysis and dimensionality reduction. Clustering is usually used in all fields to merge the same feature objects into a single group. The clustering method is incorporated with search algorithms to search the dataset from the databases. For large databases, there is a need of good clustering algorithm with high accuracy. Despite its high performance, the existing methods show some limitations. This paper focused on optimize the clustering method with a search structure for large multidimensional databases with dynamic indexes for the effective validation of construction projects. In the construction project validation to improve the clustering methods using Evolutionary Constrained Differential Optimization (ECDO) and Distributed Weighted Fuzzy C-Means algorithm to improve clustering performance than the centralized clustering approaches. The proposed methodology includes ECDO which presents new cluster tree indexing approach with cluster speed improvement in the validation of the construction projects. Distributed Fuzzy Possibilistic C-Means algorithm is proposed in improvement of cluster centre to overcome coincidence cluster problems. The comparative analysis stated that proposed ECDO model exhibits ~6% reduced memory utilization ~11% reduced scalability and ~8% minimized selectivity than the conventional WPCM model.

Structural connectome alterations between individuals with autism and neurotypical controls using feature representation learning

Autism spectrum disorder is one of the most common neurodevelopmental conditions associated with sensory and social communication impairments. Previous neuroimaging studies reported that atypical nodal- or network-level functional brain organization in individuals with autism was associated with autistic behaviors. Although dimensionality reduction techniques have the potential to uncover new biomarkers, the analysis of whole-brain structural connectome abnormalities in a low-dimensional latent space is underinvestigated. In this study, we utilized autoencoder-based feature representation learning for diffusion magnetic resonance imaging-based structural connectivity in 80 individuals with autism and 61 neurotypical controls that passed strict quality controls. We generated low-dimensional latent features using the autoencoder model for each group and adopted an integrated gradient approach to assess the contribution of the input data for predicting latent features during the encoding process. Subsequently, we compared the integrated gradient values between individuals with autism and neurotypical controls and observed differences within the transmodal regions and between the sensory and limbic systems. Finally, we identified significant associations between integrated gradient values and communication abilities in individuals with autism. Our findings provide insights into the whole-brain structural connectome in autism and may help identify potential biomarkers for autistic connectopathy.

Adaptive Wavelet Domain Principal Component Analysis for Nonstationary Time Series

High-dimensional multivariate nonstationary time series, that is, data whose second order properties vary over time, are common in many scientific and industrial applications. In this article we propose a novel wavelet domain dimension reduction technique for nonstationary time series. By constructing a time-scale adaptive principal component analysis of the data, our proposed method is able to capture the salient dynamic features of the multivariate time series. We also introduce a new time and scale dependent cross-coherence measure to quantify the extent of association between a multivariate nonstationary time series and its proposed wavelet domain principal component representation. Theoretical results establish that our associated estimation scheme enjoys good bias and consistency properties when determining wavelet domain principal components of input data. The proposed method is illustrated using extensive simulations and we demonstrate its applicability on a real-world dataset arising in a neuroscience study. Supplementary materials, with proofs of theoretical results, additional simulations and code, are available online.

A Novel Ensemble Model for Brain Tumor Diagnosis

The spread of brain tumors resulted in numerous deaths, and cancer patients are still being treated. Four novel models are introduced and compared in this study. The best one is the PIEnsemble model, which was created to correctly identify and classify MRI images for brain tumor classification. The PIEnsemble primarily combines three deep learning techniques, ResNet Model, GoogleNet Model, and Inception Model, integrated with two dimensionality reduction techniques, PCA and ICA, which are used for feature extraction with a combination of Linear, Batch1Norm1D, and ReLU layers. Simulations in a series on two benchmarks datasets were run to show the improved effectiveness of the PIEnsemble model. The experimental results highlighted the improvements to the PIEnsemble classifier structure, which has the highest classification accuracy ratio and superior performance to other methods, with a 98.25% accuracy for Dataset 1 and a 98.75% accuracy for Dataset 2.

Domain Decomposition Strategy for Combining Nonlinear and Linear Reduced-Order Models

Increasingly ubiquitous reliance on expensive, high-fidelity numerical simulations has led to the emergence of reduced-order modeling as an effective method to predict high-dimensional, spatially discretized field outputs in an efficient manner. This study presents a method to construct parametric, nonintrusive reduced-order models (ROMs) by leveraging domain decomposition (DD) to enable using multiple dimension reduction (DR) techniques within different spatial subdomains of the field to construct a single predictive ROM. To enforce smoothness of solutions across subdomains, points at domain interfaces are reconstructed using the data-driven gappy proper orthogonal decomposition method. The method is assessed on three high-dimensional test cases with shocks, including one with two-dimensional turbulent flow around the RAE2822. Results show that fully nonlinear ROMs predict fields more accurately in the vicinity of discontinuous features compared to their linear counterparts, which are more accurate in regions that are far from these discontinuities. Furthermore, irrespective of the DR method, DD-based ROMs outperformed their global counterparts in all test cases. Finally, models that used a combination of DR methods simultaneously showed superior performance near shockwaves and a significant improvement in total error compared to existing approaches that employ a single DR method in all subdomains.

Interpretable feature extraction and dimensionality reduction in ESM2 for protein localization prediction.

As the application of large language models (LLMs) has broadened into the realm of biological predictions, leveraging their capacity for self-supervised learning to create feature representations of amino acid sequences, these models have set a new benchmark in tackling downstream challenges, such as subcellular localization. However, previous studies have primarily focused on either the structural design of models or differing strategies for fine-tuning, largely overlooking investigations into the nature of the features derived from LLMs. In this research, we propose different ESM2 representation extraction strategies, considering both the character type and position within the ESM2 input sequence. Using model dimensionality reduction, predictive analysis and interpretability techniques, we have illuminated potential associations between diverse feature types and specific subcellular localizations. Particularly, the prediction of Mitochondrion and Golgi apparatus prefer segments feature closer to the N-terminal, and phosphorylation site-based features could mirror phosphorylation properties. We also evaluate the prediction performance and interpretability robustness of Random Forest and Deep Neural Networks with varied feature inputs. This work offers novel insights into maximizing LLMs' utility, understanding their mechanisms, and extracting biological domain knowledge. Furthermore, we have made the code, feature extraction API, and all relevant materials available at https://github.com/yujuan-zhang/feature-representation-for-LLMs.

Comparing dimensionality reduction techniques for visual analysis of the LSTM hidden activity on multi-dimensional time series modeling

Comparing dimensionality reduction techniques for visual analysis of the LSTM hidden activity on multi-dimensional time series modeling

Activation-Based Pruning of Neural Networks

We present a novel technique for pruning called activation-based pruning to effectively prune fully connected feedforward neural networks for multi-object classification. Our technique is based on the number of times each neuron is activated during model training. We compare the performance of activation-based pruning with a popular pruning method: magnitude-based pruning. Further analysis demonstrated that activation-based pruning can be considered a dimensionality reduction technique, as it leads to a sparse low-rank matrix approximation for each hidden layer of the neural network. We also demonstrate that the rank-reduced neural network generated using activation-based pruning has better accuracy than a rank-reduced network using principal component analysis. We provide empirical results to show that, after each successive pruning, the amount of reduction in the magnitude of singular values of each matrix representing the hidden layers of the network is equivalent to introducing the sum of singular values of the hidden layers as a regularization parameter to the objective function.

Automated Hyperspectral Feature Selection and Classification of Wildlife Using Uncrewed Aerial Vehicles

Timely and accurate detection and estimation of animal abundance is an important part of wildlife management. This is particularly true for invasive species where cost-effective tools are needed to enable landscape-scale surveillance and management responses, especially when targeting low-density populations residing in dense vegetation and under canopies. This research focused on investigating the feasibility and practicality of using uncrewed aerial systems (UAS) and hyperspectral imagery (HSI) to classify animals in the wild on a spectral—rather than spatial—basis, in the hopes of developing methods to accurately classify animal targets even when their form may be significantly obscured. We collected HSI of four species of large mammals reported as invasive species on islands: cow (Bos taurus), horse (Equus caballus), deer (Odocoileus virginianus), and goat (Capra hircus) from a small UAS. Our objectives of this study were to (a) create a hyperspectral library of the four mammal species, (b) study the efficacy of HSI for animal classification by only using the spectral information via statistical separation, (c) study the efficacy of sequential and deep learning neural networks to classify the HSI pixels, (d) simulate five-band multispectral data from HSI and study its effectiveness for automated supervised classification, and (e) assess the ability of using HSI for invasive wildlife detection. Image classification models using sequential neural networks and one-dimensional convolutional neural networks were developed and tested. The results showed that the information from HSI derived using dimensionality reduction techniques were sufficient to classify the four species with class F1 scores all above 0.85. The performances of some classifiers were capable of reaching an overall accuracy over 98%and class F1 scores above 0.75, thus using only spectra to classify animals to species from existing sensors is feasible. This study discovered various challenges associated with the use of HSI for animal detection, particularly intra-class and seasonal variations in spectral reflectance and the practicalities of collecting and analyzing HSI data over large meaningful areas within an operational context. To make the use of spectral data a practical tool for wildlife and invasive animal management, further research into spectral profiles under a variety of real-world conditions, optimization of sensor spectra selection, and the development of on-board real-time analytics are needed.

Perturbation analysis on PCA plus graph embedding methods and PCA plus exponential graph embedding methods

PCA plus graph embedding (PCA+GE) methods are popular techniques for dimensionality reduction and face recognition. In these methods, the principal component analysis (PCA) is used first, and then some graph embedding methods are applied. In practice, the data is often perturbed or contaminated, and it has been found that PCA is sensitive to noise, which may affect the accuracy of the subsequent classification. Moreover, the PCA+GE methods can be unstable after data perturbation. To the best of our knowledge, however, there are few results on perturbation analysis of this type of methods. In this work, we show the reason from a matrix perturbation analysis point of view. The main aim of this paper is to look for the reasons that cause the instability. To overcome the difficulty of instability, a framework of PCA plus exponential graph embedding methods is proposed to take the place of the conventional PCA plus graph embedding methods. The computational cost of the proposed methods is comparable to that of the original counterparts, and our methods are much more stable in terms of recognition accuracy. Numerical experiments show the effectiveness of our theoretical results, and demonstrate the efficiency of the new methods on some real-world data sets.

Bayesian tomography using polynomial chaos expansion and deep generative networks

SUMMARY Implementations of Markov chain Monte Carlo (MCMC) methods need to confront two fundamental challenges: accurate representation of prior information and efficient evaluation of likelihood functions. The definition and sampling of the prior distribution can often be facilitated by standard dimensionality-reduction techniques such as Principal Component Analysis (PCA). Additionally, PCA-based decompositions can enable the implementation of accurate surrogate models, for instance, based on polynomial chaos expansion (PCE). However, intricate geological priors with sharp contrasts may demand advanced dimensionality-reduction techniques, such as deep generative models (DGMs). Although suitable for prior sampling, these DGMs pose challenges for surrogate modelling. In this contribution, we present a MCMC strategy that combines the high reconstruction performance of a DGM in the form of a variational autoencoder with the accuracy of PCA–PCE surrogate modelling. Additionally, we introduce a physics-informed PCA decomposition to improve accuracy and reduce the computational burden associated with surrogate modelling. Our methodology is exemplified in the context of Bayesian ground-penetrating radar traveltime tomography using channelized subsurface structures, providing accurate reconstructions and significant speed-ups, particularly when the computation of the full-physics forward model is costly.

Fusion and classification algorithm of octacalcium phosphate production based on XRD and FTIR data

The present manuscript tested an automated analysis sequence to provide a decision support system to track the OCP synthesis from α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\alpha$$\\end{document}-TCP over time. Initially, the XRD and FTIR signals from a hundredfold scaled-up hydrolysis of OCP from α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\alpha$$\\end{document}-TCP were fused and modeled by the curve fitting based on the significantly established maxima from the literature and nine features extracted from the fitted shapes. Afterward, the analysis sequence enclosed the machine learning techniques for feature ranking, spatial filtering, and dimensionality reduction to support the automatic recognition of the synthesis stages. The proposed analysis pipeline for OCP identification might be the foundation for a decision support system explicitly targeting OCP synthesis. Future projects will exploit the suggested methodology for pinpointing the OCP production over time (including the intermediary phases present in the OCP formation) and for evaluating whether biological variables might be merged with biomaterial properties to build a unified model of tissue response to the implant.

Model reduction for stochastic systems with nonlinear drift

In this paper, we study dimension reduction techniques for large-scale controlled stochastic differential equations (SDEs). The drift of the considered SDEs contains a polynomial term satisfying a one-sided growth condition. Such nonlinearities in high dimensional settings occur, e.g., when stochastic reaction diffusion equations are discretized in space. We provide a brief discussion around existence, uniqueness and stability of solutions. (Almost) stability then is the basis for new concepts of Gramians that we introduce and study in this work. With the help of these Gramians, dominant subspace is identified leading to a balancing related highly accurate reduced order SDE. We provide an algebraic error criterion and an error analysis of the propose model reduction schemes. The paper is concluded by applying our method to spatially discretized reaction diffusion equations.

Mathematical scheduling model of complex industrial process combining swarm intelligence algorithm and swarm dimension reduction technology

The swarm intelligence algorithm is widely used in scheduling problems due to its superior searchability and adaptability, but it faces high computational costs and low efficiency when dealing with high-dimensional complex data. To enhance the handling of large-scale and intricate data, as well as enhance the accuracy and efficiency of scheduling, this study develops a mathematical model for industrial scheduling. The model is based on swarm intelligence algorithms such as symbiotic evolution algorithms, grey wolf algorithms, cuckoo algorithms, population dimensionality reduction techniques, and resource optimal allocation calculation methods. The final test results showed that the accuracy of the model in calculating the objective function was about 80 % in cases of high complexity and about 90 % in cases of low complexity. Compared with other algorithms, its accuracy was improved by 7 %, error rate was reduced by 50 %, time cost was reduced by about 30 %, and conversion rate was fast and more stable on average after 200 generations. The experimental results indicate that the proposed mathematical scheduling model, which integrates multiple intelligent algorithms and population dimensionality reduction technologies, has certain practices and feasibility in large-scale and multi-dimensional production for enterprises. This study combines swarm intelligence algorithms with population dimensionality reduction techniques, which is an innovative attempt at complex industrial scheduling problems.

Enhancing Sales Demand Prediction for Supply Chain Management: A Dimensionality Reduction Approach at Mukwano Company Limited, Uganda

In the dynamic landscape of today's business environment, the accurate prediction of sales demand plays a pivotal role for companies striving to maintain competitiveness. This study focuses on refining demand sales prediction for supply chain management at Mukwano Industries Limited through the application of dimensionality reduction techniques. The research involved a systematic iterative process encompassing problem identification, data preparation, modeling, evaluation, validation, optimization, and documentation. To ensure confidentiality, secondary data from the Mukwano IT department underwent meticulous merging and anonymization. Four dimensionality reduction algorithms, namely PCA, SVD, MDS, and t-SNE, were employed and evaluated using RMSE metrics. The results reveal that MDS and t-SNE exhibited exceptional performance, achieving accuracies of 89% and 88.8%, respectively. PCA and SVD also demonstrated commendable performance with an accuracy of 82.4%. The study underscores the crucial role of dimensionality reduction in enhancing predictive accuracy and optimizing inventory management. Recommendations include the incorporation of feature selection and regularization techniques to address the challenges associated with the curse of dimensionality. In conclusion, this research contributes valuable insights into the effectiveness of diverse dimensionality reduction techniques for demand prediction and inventory management. Additionally, the study highlights the need to address the curse of dimensionality and suggests exploring further research avenues in other aspects of supply chain management. These recommendations are essential for guiding future research efforts in this evolving field. Keywords: Demand sales prediction, Supply chain management, Feature selection, Inventory management, Predictive accuracy

Dimension reduction, exact recovery, and error estimates for sparse reconstruction in phase space

An important theme in modern inverse problems is the reconstruction of time-dependent data from only finitely many measurements. To obtain satisfactory reconstruction results in this setting it is essential to strongly exploit temporal consistency between the different measurement times. The strongest consistency can be achieved by reconstructing data directly in phase space, the space of positions and velocities. However, this space is usually too high-dimensional for feasible computations. We introduce a novel dimension reduction technique, based on projections of phase space onto lower-dimensional subspaces, which provably circumvents this curse of dimensionality: Indeed, in the exemplary framework of superresolution we prove that known exact reconstruction results stay true after dimension reduction, and we additionally prove new error estimates of reconstructions from noisy data in optimal transport metrics which are of the same quality as one would obtain in the non-dimension-reduced case.

Nonlinear dimensionality reduction based visualization of single-cell RNA sequencing data

Single-cell multi-omics technology has catalyzed a transformative shift in contemporary cell biology, illuminating the nuanced relationship between genotype and phenotype. This paradigm shift hinges on the understanding that while genomic structures remain uniform across cells within an organism, the expression patterns dictate physiological traits. Leveraging high throughput sequencing, single-cell RNA sequencing (scRNA-seq) has emerged as a powerful tool, enabling comprehensive transcriptomic analysis at unprecedented resolution. This paper navigates through a landscape of dimensionality reduction techniques essential for distilling meaningful insights from the scRNA-seq datasets. Notably, while foundational, Principal Component Analysis may fall short of capturing the intricacies of diverse cell types. In response, nonlinear techniques have garnered traction, offering a more nuanced portrayal of cellular relationships. Among these, Pairwise Controlled Manifold Approximation Projection (PaCMAP) stands out for its capacity to preserve local and global structures. We present an augmented iteration, Compactness Preservation Pairwise Controlled Manifold Approximation Projection (CP-PaCMAP), a novel advancement for scRNA-seq data visualization. Employing benchmark datasets from critical human organs, we demonstrate the superior efficacy of CP-PaCMAP in preserving compactness, offering a pivotal breakthrough for enhanced classification and clustering in scRNA-seq analysis. A comprehensive suite of metrics, including Trustworthiness, Continuity, Mathew Correlation Coefficient, and Mantel test, collectively validate the fidelity and utility of proposed and existing techniques. These metrics provide a multi-dimensional evaluation, elucidating the performance of CP-PaCMAP compared to other dimensionality reduction techniques.

An integrated grey-box model for accurate ship engine performance prediction under varying speed and environmental conditions

The development of ship engine fault diagnosis has led to an increased interest in predicting marine engine performance under various environmental and operating conditions. However, predicting engine performance using a single model is limited due to the various characteristics of ship engines depending on speed and environmental conditions. To address this issue, we propose a grey box model (GBM)-based ship engine performance prediction framework that combines the white box model and black box model (BBM) appropriately for both low- and high-speed operating conditions. The application of data preprocessing techniques, such as clustering and cleaning, under specific engine and environmental conditions, combined with dimensionality reduction techniques (partial least square and principal component) and the use of WBM/BBM models under classified speed conditions, makes the proposed framework accessible to real-world operators through data-driven approaches and domain knowledge. We demonstrate that the proposed framework improves the robustness, accuracy, and efficiency of engine performance predictions by considering the characteristics of each speed in real-world navigation data, compared to using single models.

High-density transparent graphene arrays for predicting cellular calcium activity at depth from surface potential recordings.

Optically transparent neural microelectrodes have facilitated simultaneous electrophysiological recordings from the brain surface with the optical imaging and stimulation of neural activity. A remaining challenge is to scale down the electrode dimensions to the single-cell size and increase the density to record neural activity with high spatial resolution across large areas to capture nonlinear neural dynamics. Here we developed transparent graphene microelectrodes with ultrasmall openings and a large, transparent recording area without any gold extensions in the field of view with high-density microelectrode arrays up to 256 channels. We used platinum nanoparticles to overcome the quantum capacitance limit of graphene and to scale down the microelectrode diameter to 20 µm. An interlayer-doped double-layer graphene was introduced to prevent open-circuit failures. We conducted multimodal experiments, combining the recordings of cortical potentials of microelectrode arrays with two-photon calcium imaging of the mouse visual cortex. Our results revealed that visually evoked responses are spatially localized for high-frequency bands, particularly for the multiunit activity band. The multiunit activity power was found to be correlated with cellular calcium activity. Leveraging this, we employed dimensionality reduction techniques and neural networks to demonstrate that single-cell and average calcium activities can be decoded from surface potentials recorded by high-density transparent graphene arrays.