Dimensionality Reduction Scheme Research Articles

A leader-adaptive particle swarm optimization with dimensionality reduction strategy for feature selection

Feature selection (FS) is a key data pre-processing method in machine learning tasks. It aims to obtain better classification accuracy of an algorithm with the smallest size of selected feature subset. Particle Swarm Optimization has been widely applied in FS tasks. However, when solving FS task on high-dimensional datasets, most of the PSO-based FS methods are easy to get premature convergence and fall into the local optimum. To address this issue, a leader-adaptive particle swarm optimization with dimensionality reduction strategy (LAPSO-DR) is proposed in this paper. Firstly, a hybrid initialization strategy based on feature importance is formulated. The population is divided into two parts, which have different initialization ranges. It can not only improve the diversity of the population but also eliminate some redundant features. Secondly, the leader-adaptive strategy is proposed to improve the exploitation ability of the population, in which each particle can have a different learning exemplar selected from the elite sub-swarm. Finally, the dimensionality reduction strategy based on Markov blanket is introduced to reduce the size of the optimal feature subset. LAPSO-DR is compared with 8 representative FS methods on 18 benchmark datasets. The experimental results show that LAPSO-DR can obtain smaller sizes of feature subsets with highest classification accuracies on 17 out of 18 datasets. The classification accuracies of LAPSO-DR are over 90% on 14 datasets and the feature elimination rates are higher than 60% on 18 datasets.

Black-box adversarial patch attacks using differential evolution against aerial imagery object detectors

Adversarial patch attacks are commonly used to attack aerial imagery object detectors. However, most existing methods are designed for white-box settings, which are impractical in real world scenarios. In this work, we propose a novel framework for black-box adversarial patch attacks against aerial imagery object detectors using differential evolution (DE). Specifically, we first introduce a new dimensionality reduction strategy to address the high-dimensionality curse in DE optimization problems. Then, we design three universal fitness functions to help DE to select better individuals within a limited computational budget. Finally, we conduct extensive experiments on the Dataset for Object Detection in Aerial images (DOTA) against You Only Look Once (YOLOv3, YOLOv4), and Faster Region-based Convolutional Network (Faster R-CNN), utilizing attack success rate (ASR) and average precision (AP) to quantitatively evaluate the attack efficiency. Results on YOLOv3 for the plane category indicate that our proposed strategy achieves at least a 20.97% improvement in ASR compared to existing dimensionality reduction schemes. Moreover, experiments conducted on detectors with different architectures reveal inconsistent adversarial robustness across categories. To the best of our knowledge, this is the first work to explore the use of DE in black-box adversarial patch attacks against aerial imagery object detectors. Our code has been released at https://github.com/tang-agui/Bbox-Att-Detector.

Autocleandeepfood: auto-cleaning and data balancing transfer learning for regional gastronomy food computing

AbstractFood computing has emerged as a promising research field, employing artificial intelligence, deep learning, and data science methodologies to enhance various stages of food production pipelines. To this end, the food computing community has compiled a variety of data sets and developed various deep-learning architectures to perform automatic classification. However, automated food classification presents a significant challenge, particularly when it comes to local and regional cuisines, which are often underrepresented in available public-domain data sets. Nevertheless, obtaining high-quality, well-labeled, and well-balanced real-world labeled images is challenging since manual data curation requires significant human effort and is time-consuming. In contrast, the web has a potentially unlimited source of food data but tapping into this resource has a good chance of corrupted and wrongly labeled images. In addition, the uneven distribution among food categories may lead to data imbalance problems. All these issues make it challenging to create clean data sets for food from web data. To address this issue, we present AutoCleanDeepFood, a novel end-to-end food computing framework for regional gastronomy that contains the following components: (i) a fully automated pre-processing pipeline for custom data sets creation related to specific regional gastronomy, (ii) a transfer learning-based training paradigm to filter out noisy labels through loss ranking, incorporating a Russian Roulette probabilistic approach to mitigate data imbalance problems, and (iii) a method for deploying the resulting model on smartphones for real-time inferences. We assess the performance of our framework on a real-world noisy public domain data set, ETH Food-101, and two novel web-collected datasets, MENA-150 and Pizza-Styles. We demonstrate the filtering capabilities of our proposed method through embedding visualization of the feature space using the t-SNE dimension reduction scheme. Our filtering scheme is efficient and effectively improves accuracy in all cases, boosting performance by 0.96, 0.71, and 1.29% on MENA-150, ETH Food-101, and Pizza-Styles, respectively.

Hyperspectral remote sensing of forage stoichiometric ratios in the senescent stage of alpine grasslands

Senescence is an important phenological stage in the life cycle of grassland plants, and estimating grass senescence in alpine grasslands is particularly crucial for animal husbandry and grazing the utilization of grassland resources for grazing. However, the application of proximal sensing technologies to capture the variations in forage growth parameters in the senescent stage, especially regarding some key stoichiometric ratios, still faces challenges associated with indiscernible spectral properties and indeterminate absorption and reflectance features. The present study aims to demonstrate the potential and effectiveness of applying hyperspectral data to assess the forage carbon-nitrogen (C:N) ratio and calcium-phosphorus (Ca:P) ratio during the senescent stage (September to November). A progressive variable dimensionality reduction strategy based on a genetic algorithm (GA) and hybrid optimization method (ant optimization colony (ACO) and extreme learning machine (ELM)) coupled with multiple linear and nonlinear statistical methods is applied to develop C:N ratio and Ca:P ratio estimation models via combination with data from 205 sample sites collected from six field campaigns (2016–2019) on the eastern Tibetan Plateau. The results show that the important spectral bands favorable for forage C:N ratio and Ca:P ratio retrieval are generally in the red and shortwave infrared (SWIR) regions, and the proposed model presents satisfactory performance in the estimation of the forage C:N ratio (V-R2 = 0.81, V-RMSE = 6.44) and Ca:P ratio (V-R2 = 0.64, V-RMSE = 2.54) during senescence. Moreover, the model can effectively overcome the spatial differences among sampling areas and perform optimally in capturing the variations in the forage C:N ratio and Ca:P ratio in the early and middle stages of senescence. Overall, our study demonstrates that it is feasible and promising to estimate these stoichiometric ratios using hyperspectral feature bands during grass senescence at the canopy level, with potential applications that may further enhance the monitoring of forage quality and quantity.

A comprehensive benchmarking of machine learning algorithms and dimensionality reduction methods for drug sensitivity prediction.

A major challenge of precision oncology is the identification and prioritization of suitable treatment options based on molecular biomarkers of the considered tumor. In pursuit of this goal, large cancer cell line panels have successfully been studied to elucidate the relationship between cellular features and treatment response. Due to the high dimensionality of these datasets, machine learning (ML) is commonly used for their analysis. However, choosing a suitable algorithm and set of input features can be challenging. We performed a comprehensive benchmarking of ML methods and dimension reduction (DR) techniques for predicting drug response metrics. Using the Genomics of Drug Sensitivity in Cancer cell line panel, we trained random forests, neural networks, boosting trees and elastic nets for 179 anti-cancer compounds with feature sets derived from nine DR approaches. We compare the results regarding statistical performance, runtime and interpretability. Additionally, we provide strategies for assessing model performance compared with a simple baseline model and measuring the trade-off between models of different complexity. Lastly, we show that complex ML models benefit from using an optimized DR strategy, and that standard models-even when using considerably fewer features-can still be superior in performance.

Hybrid control strategy for efficiency enhancement of a raft-type wave energy converter

To improve the wave energy capture performance of a raft-type wave energy converter, a novel hybrid control strategy consisting of a passive control strategy and an active control strategy is proposed. In the passive control strategy, a nonlinear stiffness mechanism is introduced to generate a nonlinear passive control force. The passive control strategy has the advantage of broadening the capture width, whereas the active control strategy, which uses nonlinear model predictive control, is mainly utilized to improve the capture efficiency. Considering the high dimensions of the dynamic model, a dimension reduction strategy is given. Moreover, a mechanism for selecting the initial value is proposed to avoid the local optimum. Then, a dimension reduction nonlinear model predictive control controller is proposed to obtain the optimal active control force. Finally, the wave energy capture performance is evaluated for both regular and irregular waves. Numerical simulation results show that the wave energy capture performance can be improved using the novel hybrid control strategy.

Exploring machine learning strategies for predicting cardiovascular disease risk factors from multi-omic data

BackgroundMachine learning (ML) classifiers are increasingly used for predicting cardiovascular disease (CVD) and related risk factors using omics data, although these outcomes often exhibit categorical nature and class imbalances. However, little is known about which ML classifier, omics data, or upstream dimension reduction strategy has the strongest influence on prediction quality in such settings. Our study aimed to illustrate and compare different machine learning strategies to predict CVD risk factors under different scenarios.MethodsWe compared the use of six ML classifiers in predicting CVD risk factors using blood-derived metabolomics, epigenetics and transcriptomics data. Upstream omic dimension reduction was performed using either unsupervised or semi-supervised autoencoders, whose downstream ML classifier performance we compared. CVD risk factors included systolic and diastolic blood pressure measurements and ultrasound-based biomarkers of left ventricular diastolic dysfunction (LVDD; E/e' ratio, E/A ratio, LAVI) collected from 1,249 Finnish participants, of which 80% were used for model fitting. We predicted individuals with low, high or average levels of CVD risk factors, the latter class being the most common. We constructed multi-omic predictions using a meta-learner that weighted single-omic predictions. Model performance comparisons were based on the F1 score. Finally, we investigated whether learned omic representations from pre-trained semi-supervised autoencoders could improve outcome prediction in an external cohort using transfer learning.ResultsDepending on the ML classifier or omic used, the quality of single-omic predictions varied. Multi-omics predictions outperformed single-omics predictions in most cases, particularly in the prediction of individuals with high or low CVD risk factor levels. Semi-supervised autoencoders improved downstream predictions compared to the use of unsupervised autoencoders. In addition, median gains in Area Under the Curve by transfer learning compared to modelling from scratch ranged from 0.09 to 0.14 and 0.07 to 0.11 units for transcriptomic and metabolomic data, respectively.ConclusionsBy illustrating the use of different machine learning strategies in different scenarios, our study provides a platform for researchers to evaluate how the choice of omics, ML classifiers, and dimension reduction can influence the quality of CVD risk factor predictions.

Fully unsupervised wear anomaly assessment of aero-bearings enhanced by multi-representation learning of deep features

The assessment of wear anomalies using vision-based techniques is crucial for automated inspections of machine health conditions. Given the diversity of worn anomalies and the time-consuming nature of labeling, recent advancements in unsupervised anomaly detection have focused on comparing extracted features with those of normal images. However, this approach faces significant challenges in constructing a comprehensive comparison sample database and extracting distinctive features from worn surfaces with varying degrees of severity. To overcome these limitations, we introduce a fully unsupervised anomaly assessment method that detects anomalies solely based on single-bearing images, without the need for unworn samples. This approach eliminates the requirement for a comparison sample database by constructing an implicit feature database from the single-bearing surfaces using pre-trained convolutional neural networks and dimensionality reduction strategies. Additionally, to address the limited differentiability of wear characteristics, we develop a feature refinement strategy that incorporates multi-representation learning and topographical distribution characteristics of worn surfaces. By integrating the statistics of the refined feature database with uncertainty distribution-based normal feature weights, we construct an anomaly distribution map. This map enables us to assess the anomaly degree of a single-bearing effectively. To validate our method, we conducted experiments using real aero-engine bearings. The results demonstrate that our proposed approach can accurately assess anomaly degrees without pre-prepared comparison samples, achieving a detection performance of 90%, which fulfills the requirements for wear anomaly assessment.

A Bayesian Structural Modal Updating Method Based on Sparse Grid and Ensemble Kalman Filter

This study presents a sparse grid interpolation and ensemble Kalman filter (EnKF)-based Markov Chain Monte Carlo (MCMC) method (SG-EnMCMC). Initiating with the formulation of a recursive equation for the state space vector, derived from the structural dynamic equation, this study adopts a dimensionality reduction strategy. This approach involves a separation of physical parameters and the state space vector. The acquisition of physical parameters is accomplished through sampling, utilizing sample moments to substitute population moments, thereby mitigating the need for computationally high-dimensional covariance matrix calculations. To further streamline the recursive equation of the state space vector, a sparse grid method is employed for interpolation. This step simplifies the process while ensuring superior accuracy compared to the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF). Subsequent to this, acceptance rates and the final parameter posterior distribution within the MCMC framework are derived. The efficiency of the proposed method is assessed through validation in two shaking table experiments.

Dimensional reduction technique-based maximum entropy principle method for safety degree analysis under twofold random uncertainty

A modified failure chance measure (FCM) was proposed to assess the safety degree of structures under the influence of twofold random uncertainty. This uncertainty arises from random inputs with random distribution parameters. The aim of this paper is to effectively evaluate the safety degree of structures in such conditions. This paper introduces a method named dimensional reduction technique-based maximum entropy principle to address the issue at hand. The proposed method utilizes maximum entropy principle method to efficiently approach optimal probability density characteristics while adhering to the constraints imposed by fractional moments. Additionally, the dimensional reduction strategy is employed to estimate fractional moments, resulting in a linear increase in computational cost with respect to the dimensionality. The primary contribution of this work involves the detailed decoupling of the double-uncertainty analysis used to estimate FCM into a single-uncertainty analysis. This approach allows for the innovative re-use of the same group integral grid points to estimate different fractional moments required for solving FCM. The results of applying the proposed method to solve FCM under acceptable accuracy demonstrate that the number of evaluations required for the performance function can be reduced to less than 100 when the uncertainty dimensionality is limited to 20. This finding confirms the high efficiency of the proposed method for solving FCM.

Soft computing methods in the solution of an inverse heat transfer problem with phase change: A comparative study

Inverse heat transfer problems are ill-posed problems and their solution is challenging. Conventional (hard computing) solution methods were developed for this purpose in the past, but they are not well applicable in cases including phase change, which contain strong non-linearity and bring additional computational difficulties. Soft computing methods, which currently experience very rapid development, are a promising tool for the solution of such problems. This paper addresses an inverse heat transfer problem with phase change, in which the boundary heat flux is estimated. Four methods based on distinct mathematical principles are applied to this problem and thoroughly compared. These methods include a conventional Levenberg–Marquardt method (LMM), a predictive fuzzy logic (PFL)-based method, a population-based meta-heuristic method called LSHADE (a state-of-the-art differential evolution variant), and a recently developed surrogate-assisted method coupled with differential evolution, referred to as LSADE method. Furthermore, a reformulation of the problem was developed, utilising a dimension reduction scheme and a decomposition scheme that led to sub-problems with different time frames. This reformulation brought extensive computational improvements. Results of the comparison of the methods then showed that the LMM and the PFL behave well in case without phase change but their performance deteriorates substantially in case with phase change. The LSHADE and the LSADE showed superior performance in the solution of the inverse problem with the phase change. Moreover, their performance was rather stable and insensitive to the location of the temperature sensor, which was the source of data for the estimation.

Mining the High Dimensional Biological Dataset Using Optimized Colossal Pattern with Dimensionality Reduction

Recent years have seen a lot of attention paid to the mining of enormous item sets from high-dimensional databases. Small and mid-sized data sets take a long time to mine with traditional algorithms since they don’t include the complete and relevant info needed for decision making. Many applications, particularly in bioinformatics, benefit greatly from the extraction of (FCCI) Frequent Colossal Closed Itemsets from a large dataset. In order to extract FCCI from a dataset, present preprocessing strategies fail to remove all extraneous characteristics and rows from the data set completely. In addition, the most current algorithms for this kind are sequential and computationally expensive. A high-dimensional dataset is pruned of all extraneous characteristics and rows using two alternative dimensionality reduction strategies presented in this paper. Then, an optimal feature value is identified by using Equilibrium Optimizer (EO) to identify the threshold value for reduced features. It is designed to discover common items and build association rules if the feature value is smaller than the frequency mining algorithm (IFRS) in conjunction with the Fruit fly Algorithm (FFA). If the feature value exceeds the optimal threshold, then optimized Length restrictions can be used to solve the CP mining problem (LC). Random search is utilized to identify the optimal threshold values of the restrictions and extract the enormous pattern using the Differential Evolutionary Arithmetic Optimization Algorithm. The experiments are carried on twenty biological datasets that us extracted from UCI websites and validated the proposed models in terms of various metrics.

A novel approach for detecting sensor-based semiconductor fault yield classification using convolutional neural networks

<p>In the proposed research, data from the semiconductor industry is considered for analysis. In this research, there is a requirement for significantly more space for storage, processing will take significantly more time, and there will be a significant amount of duplicate data. So, the utilization of dimensionality reduction strategies is required so as to lessen the number of spectral bands while maintaining the maximum amount of relevant information. Our contribution can be broken down into two parts: To begin, we suggest a filter-based technique that we call interband redundancy analysis (IBRA). This method is based on a collinearity analysis that is performed among a band and its neighbors. By performing the given research, redundant bands can be omitted, which in turn significantly brings down the search space. Next, we take the findings of the IBRA and use a wrapper-based technique known as greedy spectral selection (GSS) to choose bands on the basis of the information entropy values of those bands. We are later training a convolutional neural network to evaluate how well the present selection is working. We also propose an optimization algorithm for performance enhancement known as bacterial foraging optimization.</p>

Predicting soil erosion and sediment delivery in large, data‐sparse, mountainous basins

AbstractIntense hydroelectric exploitation requires information on sediment flux to prevent the potential siltation of cascade reservoirs. As the main sediment source for the Yangtze River, the lower Jinsha River Basin (LJRB) lacks data on its sediment budget, which urges a thorough investigation to provide reliable estimates. This paper proposed a modelling framework to study soil erosion, sediment delivery, and sediment yield in this critical area with an emphasis on analyzing the underlying controls. Over the past 50 years, basin average erosion rates decreased from 26.5 to 23.7 t ha−1 a−1, with erosion hotspots (>50 t ha−1 a−1) comprising only 21.1% of the basin area but contributing 69.5% of the gross erosion. Local steepness and sparse landcover conditions were found to be directly responsible for high erosions, rather than precipitation or soils. With a dimensionality reduction strategy to remove redundant factors, the optimal variables for modelling sediment delivery ratio (SDR) were identified as Specific Catchment Area, Maximum Elevation, and Drainage Area. These three variables all had a positive correlation with SDR, and the established model can explain 86% of the SDR variation. Further modelling revealed regionalised SDRs ranging from 0.07 to 0.38 for 39 subbasins, about 30% of which were no less than 0.35. Regionalised specific sediment yield (SSY) ranged from 1.3 to 16.9 t ha−1 a−1 with 38% of the high SSYs (above 8 t ha−1 a−1) concentrating in the reservoir area from Wudongde to Xiluodu that poses siltation risks. This study addressed the scarcity of sediment budget knowledge in the LJRB, and the coupled model of erosion and sediment delivery has been validated with satisfactory predictions. Therefore, the modelling framework proposed may be adopted to diverse river basins for its low parameter requirements and solid modelling performance.

HCDR-based motion planning for mobile manipulators with the continuous trajectory task

Mobile manipulators have been widely used in modern industrial fields due to their advantages of large operating workspace and high motion dexterity. For the application, the path planning has been a core foundation for its motion control. However, as the environmental complexity and performance requirements increase, existing methods are limited by low execution efficiency, and lack of comprehensive consideration of motion performances. A motion planning technique is proposed for mobile manipulators with continuous trajectory task. In this technique, a hierarchical constraints dimension reduction (HCDR) strategy is presented based on the task decomposition and performance requirements analysis of each stage. HCDR strategy aims to improve the execution efficiency of the algorithm through local operations of performance constraints and the local simplification of the distance calculation at different stages of the task. Models of inverse kinematics and performance indexes are established to provide basis for constraint conditions, including distance models with different accuracy, manipulability index model and trajectory tracing error model. Two optimization strategies are presented to solve the motion planning problem considering the overall performance of the system. And simulation results show that the three-level strategy has the higher convergence efficiency than the bi-level strategy, as well as can guarantee the motion performance of the system.

Wind energy assessment considering a truncated distribution of probabilistic turbulence power spectral parameters

Accurate probabilistic wind spectra are beneficial to reproducing or predicting wind speeds, then assessing wind energy. The non-truncated distribution is widely-used for fitting spectral parameters. However, it is not consistent with the bounded measured parameters. Therefore, the truncated distribution, firstly, is proposed in modelling three spectral parameters in the Ma'anshan Yangtze River (MYR) Bridge. The distribution parameters conditional to wind velocities are determined through estimated extreme values under a given recurrence period, which is derived from the established joint distribution of extreme spectral parameter and wind velocities. The fitness of the truncated distribution on measured values shows better performance than the non-truncated distribution. The dimension-reduction (DR) expression of stochastic wind fields is represented by considering probability-truncated wind spectra. The simulation of stochastic wind fields at the MYR Bridge is realized based on the proposed DR scheme. The comparisons of wind velocities and wind power densities are implemented between truncated and non-truncated distributions. Results shows the feasibility of the proposed method to exhibit probability of local wind fields. Meanwhile, wind speeds and wind power densities are more significant in the truncated distribution.

Hybrid Causal Feature Selection for Cancer Biomarker Identification from RNA-seq Data.

The discovery of cancer biomarkers helps to advance medical diagnosis and plays an important role in biomedical applications. Most of the existing data-driven methods identify biomarkers by ranking-based strategies, which generally return a subset or superset of the actual biomarkers, while some other causal-wise feature selection methods are based on Markov Blanket (MB) learning, facing the challenges of high-dimensionality & low-sample. In this work, we propose a novel hybrid causal feature selection method (called CAFES) to support large-scale cancer biomarker discovery from real RNA-seq data. Concretely, CAFES first uses minimal-redundancy & maximal-relevance strategy for dimensionality reduction that returns a set of candidate features. CAFES then learns the causal skeleton w.r.t. those features by CI tests and further obtains an appropriate superset of the MB of the target variable. Finally, CAFES learns the causal structure of this superset by the DAG-GNN algorithm and then obtains the MB of the target variable, which can be treated as the cancer biomarkers. We conduct experiments to evaluate the proposed method on two real well-known RNA-seq datasets that covering both binary and multi-class cases. We compare our method CAFES with seven recent methods including Semi-HITON-MB, STMB, BAMB, FBED, LCS-FS, EEMB, and EAMB. The results show that CAFES can identify dozens of cancer biomarkers, and 1/6 ∼1/2 of the discovered biomarkers can be verified by existing works that they are really directly related to the corresponding disease. An advantage of CAFES is that its Recall is significantly higher than those of all the counterparts, indicating that the continuous optimization (DAG-GNN) with the returned causal skeleton after feature selection (that can be treated as a conditional independence-based constraint to the optimization problem) is effective in cancer biomarkers identification under high-dimensional and low-sample RNA-seq data. The source code of CAFES is available at https://github.com/Milkteaww/CFS.

Fast Bayesian Linearized Inversion With an Efficient Dimension Reduction Strategy

Fast Bayesian Linearized Inversion With an Efficient Dimension Reduction Strategy

Evaluating dimensionality reduction strategies on mixed-type datasets: a comparative analysis using Python, R and SPSS

Evaluating dimensionality reduction strategies on mixed-type datasets: a comparative analysis using Python, R and SPSS

An Optimization Method of Flexible Manufacturing System Reliability Allocation Based on Two Dimension-Reduction Strategies

As increasingly extensive applications of flexible manufacturing systems (FMSs) arise, their reliability allocation has been a research hot spot. But, since FMSs are always composed of transfer and buffer devices, production machines, and complex control systems, the large number of basic elements makes the number of variables and constraints of reliability-allocation optimization increase greatly, which leads to the difficulty and inefficiency of optimization. To solve the above problem, two dimension-reduction strategies are proposed for the FMS reliability optimization with low cost and a high level of reliability as the objectives, and they are the reliability-weight double-threshold qualification strategy (RWTS) and the bi-level optimization strategy (BLOS), respectively. Based on these two strategies, an overall reliability-allocation optimization model and a bi-level reliability-allocation optimization model are established based on the FMS reliability evaluation presented in our previous work, and their algorithms based on particle swarm optimization (PSO) are presented. In terms of their contributions, for the RWTS, thresholds of reliability and the weight index of each basic element are set to dynamically reduce the number of variables in each iteration of the optimization; for the BLOS, the overall reliability-allocation optimization problem for transitioning from the FMS to basic elements can be transformed into simpler allocation optimizations from the FMS to subsystems and from subsystems to basic elements, which have fewer variables, and this can largely improve the optimization convergence performance. Through applying this to a box-parts finishing FMS, compared with the traditional optimization method, the high efficiency and the good allocation effect of the optimization based on these two strategies for improving convergence speed are verified by the simulation results. The proposed method has great significance for FMS design due to its limited cost but high-reliability requirement.