Curse Of Dimensionality Research Articles

A segmented trial-and-error algorithm for optimized operation of water storage projects

ABSTRACT Mathematical programming methods and modern intelligent algorithms are used to solve the optimized operation models of water storage projects. Classical mathematical programming methods are prone to the curse of dimensionality when the size of the system increases. Intelligent algorithms improve the computational efficiency to a certain extent, but still suffer from defects such as unstable optimized solutions and easy to fall into local optimizers. In this article, a segmented trial-and-error algorithm is proposed to obtain the global optimizer while considering computational efficiency. First, a water balance constraint is introduced into the objective function and a Lagrangian function is constructed to derive the ideal optimizer. Constraints are then gradually introduced and the optimized model is solved by iterative trial-and-error calculations. By approximating the ideal optimizer by segments in this way, the global optimizer is finally obtained. Numerical experiments were carried out in a water supply reservoir. The results show that the segmented trial-and-error algorithm can obtain the global optimizers in different situations with objective functions that are 0.0028–0.0076 lower than those of the genetic algorithm, implying less and more uniform water shortages. The segmented trial-and-error algorithm requires less than 1% of the computational time of the dynamic programming.

Prediction of Glass Transition Temperature of Polymers Using Simple Machine Learning.

Polymer materials have garnered significant attention due to their exceptional mechanical properties and diverse industrial applications. Understanding the glass transition temperature (Tg) of polymers is critical to prevent operational failures at specific temperatures. Traditional methods for measuring Tg, such as differential scanning calorimetry (DSC) and dynamic mechanical analysis, while accurate, are often time-consuming, costly, and susceptible to inaccuracies due to random and uncertain factors. To address these limitations, the aim of the present study is to investigate the potential of Simplified Molecular Input Line Entry System (SMILES) as descriptors in simple machine learning models to predict Tg efficiently and reliably. Five models were utilized: k-nearest neighbors (KNNs), support vector regression (SVR), extreme gradient boosting (XGBoost), artificial neural network (ANN), and recurrent neural network (RNN). SMILES descriptors were converted into numerical data using either One Hot Encoding (OHE) or Natural Language Processing (NLP). The study found that SMILES inputs with fewer than 200 characters were inadequate for accurately describing compound structures, while inputs exceeding 200 characters diminished model performance due to the curse of dimensionality. The ANN model achieved the highest R2 value of 0.79; however, the XGB model, with an R2 value of 0.774, exhibited the highest stability and shorter training times compared to other models, making it the preferred choice for Tg prediction. The efficiency of the OHE method over NLP was demonstrated by faster training times across the KNN, SVR, XGB, and ANN models. Validation of new polymer data showed the XGB model's robustness, with an average prediction deviation of 9.76 from actual Tg values. These findings underscore the importance of optimizing SMILES conversion methods and model parameters to enhance prediction reliability. Future research should focus on improving model accuracy and generalizability by incorporating additional features and advanced techniques. This study contributes to the development of efficient and reliable predictive models for polymer properties, facilitating the design and application of new polymer materials.

Heatmap Design for Probabilistic Driver Repositioning in Crowdsourced Delivery

This paper proposes the use of heatmaps as a control lever to manage the probabilistic repositioning of independent drivers in crowdsourced delivery platforms. The platform aims to maximize order fulfillment by dynamically matching drivers and orders and selecting heatmaps that trigger the probabilistic flow of unmatched drivers to balance driver supply and delivery requests across the service region. We develop a Markov decision process (MDP) model to sequentially select matching and heatmap decisions in which the repositioning behavior of drivers is captured by a multinomial logit discrete choice model. Because of the curse of dimensionality and the endogenous uncertainty of driver repositioning, the MDP model is solved using a rolling-horizon stochastic lookahead policy. This policy decomposes matching and heatmap decisions into two optimization problems: a two-stage stochastic programming upper bounding problem for matching decisions and a mixed-integer programming problem for heatmap decisions. We also propose a simple policy for efficiently solving large-scale problems. An extensive computational study on instances derived from the Chicago ride-hailing data set is conducted. Computational experiments demonstrate the value of heatmaps in improving order fulfillment beyond the level achieved by matching alone (up to 25%) and identify conditions that affect the benefit of using heatmaps to guide driver repositioning. Funding: The authors gratefully acknowledge the support of the Natural Sciences and Engineering Research Council of Canada through Discovery Grants [Grants RGPIN-2024-04881, RGPIN-2020-04498, and RGPIN-2019-06207] awarded to the first, second, and third authors, respectively. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2022.0418 .

A low-rank support tensor machine for multi-classification

In recent decades, there has been an increasing demand for effectively handling high-dimensional multi-channel tensor data. Due to the inability to utilize internal structural information, Support Vector Machine (SVM) and its variations struggle to classify flattened tensor data, consequently resulting in the ‘curse of dimensionality’ issue. Furthermore, most of these methods can not directly apply to multiclass datasets. To overcome these challenges, we have developed a novel classification method called Multiclass Low-Rank Support Tensor Machine (MLRSTM). Our method is inspired by the well-established low-rank tensor hypothesis, which suggests a correlation between each channel of the feature tensor. Specifically, MLRSTM adopts the hinge loss function and introduces a convex approximation of tensor rank, the order-d Tensor Nuclear Norm (order-d TNN), in the regularization term. By leveraging the order-d TNN, MLRSTM effectively exploits the inherent structural information in tensor data to enhance generalization performance and avoid the curse of dimensionality. Moreover, we develop the Alternating Direction Method of Multipliers (ADMM) algorithm to optimize the convex problem inherent in training MLRSTM. Finally, comprehensive experiments validate the excellent performance of MLRSTM in tensor multi-classification tasks, showcasing its potential and efficacy in handling high-dimensional multi-channel tensor data.

CMA-ES-based topology optimization accelerated by spectral level-set-boundary modeling

Topology optimization commonly encounters several challenges, such as ill-posedness, grayscale issues, interdependencies among design variables, multimodality, and the curse of dimensionality. Furthermore, addressing the latter two concurrently presents considerable difficulty. In this study, we introduce a framework aimed at mitigating all the above obstacles simultaneously. The objective is to achieve optimal configurations in a notably reduced timeframe eliminating the need for the initial trial-and-error iterations. The topology optimization approach we propose is implemented via precise structural boundary modeling utilizing a body-fitted mesh generated using a Fourier series expanded level-set method. This methodology expedites the exploration of optimal solutions. We employ the covariance matrix adaptation-evolution strategy to address multimodality, thereby enhancing the optimization process. The implementation of the Fourier-series-expanded level-set method reduces the number of design variables while maintaining accuracy in finite-element analyses by replacing design variables from discretized level-set functions with the coefficients of the Fourier series expansion. To facilitate the exploration of optimal solutions, a method is also introduced for handling box constraints through an adaptive penalty function. To demonstrate the effectiveness of the proposed scheme, we address three distinct problems: mean compliance minimization, heat flux manipulation, and the control of electromagnetic wave scattering. Despite each system being governed by different equations, topology optimization method consistently yields notable acceleration in computational efficiency across all scenarios, and remarkably without requiring initial guesses.

Cross-ValidatedFunctional Generalized Partially Linear Single-Functional Index Model

In this paper, we have introduced a functional approach for approximating nonparametric functions and coefficients in the presence of multivariate and functional predictors. By utilizing the Fisher scoring algorithm and the cross-validation technique, we derived the necessary components that allow us to explain scalar responses, including the functional index, the nonlinear regression operator, the single-index component, and the systematic component. This approach effectively addresses the curse of dimensionality and can be applied to the analysis of multivariate and functional random variables in a separable Hilbert space. We employed an iterative Fisher scoring procedure with normalized B-splines to estimate the parameters, and both the theoretical and practical evaluations demonstrated its favorable performance. The results indicate that the nonparametric functions, the coefficients, and the regression operators can be estimated accurately, and our method exhibits strong predictive capabilities when applied to real or simulated data.

Ultrasonic guided wave-based probabilistic diagnostic imaging method with Single-Path-Scattering sparse reconstruction for Multi-Damage detection in composite structures

Plate-like carbon fiber composite structures are essential for equipment such as spacecraft and rail vehicles. These structures are exposed to long-term cyclic loads and harsh environmental conditions, leading to damage of unknown location and quantity. Ultrasonic guided wave (UGW) detection technology shows promise for evaluating the structural performance of composite structures due to its advantages of high precision, high speed, and real-time capabilities. However, multi-damage scattering sources interfere with and superimpose the received signals when multiple damages coincide. This interference can lead to artifacts in probability imaging methods that rely on features such as amplitude and time of flight (TOF). This paper proposes a single-path-scattering sparse reconstruction-based probabilistic diagnostic imaging (SSR-PDI) method. Firstly, a Lamb dictionary with time shift is constructed. Each actuator-receiver path is treated as a target signal, enabling the approximate separation and extraction of multiple scattering waves from different damages. It mitigates the “curse of dimensionality” associated with the sparse reconstruction (SR) method. Secondly, a new ring-shaped damage probability distribution is proposed based on the sparse representation matrix’s time mapping and amplitude weighting. The experimental results demonstrate that the SSR-PDI method effectively mitigates issues related to multi-damage detection, specifically false negatives/positives and localization errors arising from the superposition of scattering sources.

SIMpat: A Synthetic Benchmark for Similarity Metrics on Patient Representations.

Similarity and clustering tasks based on data extracted from electronic health records on the patient level suffer from the curse of dimensionality and the lack of inter-patient data comparability. Indeed, for many health institutions, there are many more variables, and ways of expressing those variables to represent patients than patients sharing the same set of data. To lower redundancy and increase interoperability one strategy is to map data to semantic-driven representations through medical knowledge graphs such as SNOMED-CT. However, patient similarity metrics based on this knowledge-graph information lack quantitative evaluation and comparisons with pure data-driven methods. The reasons are twofold, firstly, it is hard to conceptually assess and formalize a gold-standard similarity between patients resulting in poor inter-annotator agreement in qualitative evaluations. Secondly, the community has been lacking a clear benchmark to compare existing metrics developed by scientific communities coming from various fields such as ontology, data science, and medical informatics. This study proposes to leverage the known challenges of evaluating patient similarities by proposing SIMpat, a synthetic benchmark to quantitatively evaluate available metrics, based on controlled cohorts, which could later be used to assess their sensibility regarding aspects such as the sparsity of variables or specificities of patient disease patterns.

Crafting 10 Years of Statistics Explanations: Points of Significance

Points of Significance is an ongoing series of short articles about statistics in Nature Methods that started in 2013. Its aim is to provide clear explanations of essential concepts in statistics for a nonspecialist audience. The articles favor heuristic explanations and make extensive use of simulated examples and graphical explanations, while maintaining mathematical rigor. Topics range from basic, but often misunderstood, such as uncertainty and p-values, to relatively advanced, but often neglected, such as the error-in-variables problem and the curse of dimensionality. More recent articles have focused on timely topics such as modeling of epidemics, machine learning, and neural networks. In this article, we discuss the evolution of topics and details behind some of the story arcs, our approach to crafting statistical explanations and narratives, and our use of figures and numerical simulations as props for building understanding.

Is it possible to find the single nearest neighbor of a query in high dimensions?

We investigate an open question in the study of the curse of dimensionality: Is it possible to find the single nearest neighbor of a query in high dimensions? Using the notion of (in)distinguishability to examine whether the feature map of a kernel is able to distinguish two distinct points in high dimensions, we analyze this ability of a metric-based Lipschitz continuous kernel as well as that of the recently introduced Isolation Kernel. Between the two kernels, we show that only Isolation Kernel has distinguishability and it performs consistently well in four tasks: indexed search for exact nearest neighbor search, anomaly detection using kernel density estimation, t-SNE visualization and SVM classification in both low and high dimensions, compared with distance, Gaussian and three other existing kernels.

Pricing multi-asset options with tempered stable distributions

We derive methods for risk-neutral pricing of multi-asset options, when log-returns jointly follow a multivariate tempered stable distribution. These lead to processes that are more realistic than the better known Brownian motion and stable processes. Further, we introduce the diagonal tempered stable model, which is parsimonious but allows for rich dependence between assets. Here, the number of parameters only grows linearly as the dimension increases, which makes it tractable in higher dimensions and avoids the so-called “curse of dimensionality.” As an illustration, we apply the model to price multi-asset options in two, three, and four dimensions. Detailed goodness-of-fit methods show that our model fits the data very well.

Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK

AbstractQuantum computing promises a disruptive impact on machine learning algorithms, taking advantage of the exponentially large Hilbert space available. However, it is not clear how to scale quantum machine learning (QML) to industrial-level applications. This paper investigates the scalability and noise resilience of quantum generative learning applications. We consider the training performance in the presence of statistical noise due to finite-shot noise statistics and quantum noise due to decoherence to analyze the scalability of QML methods. We employ rigorous benchmarking techniques to track progress and identify challenges in scaling QML algorithms, and show how characterization of QML systems can be accelerated, simplified, and made reproducible when the QUARK framework is used. We show that QGANs are not as affected by the curse of dimensionality as QCBMs and to which extent QCBMs are resilient to noise.

Large-scale IoT attack detection scheme based on LightGBM and feature selection using an improved salp swarm algorithm

Due to the swift advancement of the Internet of Things (IoT), there has been a significant surge in the quantity of interconnected IoT devices that send and exchange vital data across the network. Nevertheless, the frequency of attacks on the Internet of Things is steadily rising, posing a persistent risk to the security and privacy of IoT data. Therefore, it is crucial to develop a highly efficient method for detecting cyber threats on the Internet of Things. Nevertheless, several current network attack detection schemes encounter issues such as insufficient detection accuracy, the curse of dimensionality due to excessively high data dimensions, and the sluggish efficiency of complex models. Employing metaheuristic algorithms for feature selection in network data represents an effective strategy among the myriad of solutions. This study introduces a more comprehensive metaheuristic algorithm called GQBWSSA, which is an enhanced version of the Salp Swarm Algorithm with several strategy improvements. Utilizing this algorithm, a threshold voting-based feature selection framework is designed to obtain an optimized set of features. This procedure efficiently decreases the number of dimensions in the data, hence preventing the negative effects of having a high number of dimensions and effectively extracting the most significant and crucial information. Subsequently, the extracted feature data is combined with the LightGBM algorithm to form a lightweight and efficient ensemble learning scheme for IoT attack detection. The proposed enhanced metaheuristic algorithm has superior performance in feature selection compared to the recent metaheuristic algorithms, as evidenced by the experimental evaluation conducted using the NSLKDD and CICIoT2023 datasets. Compared to current popular ensemble learning solutions, the proposed overall solution exhibits excellent performance on multiple key indicators, including accuracy, precision, as well as training and detection time. Especially on the large-scale dataset CICIoT2023, the proposed scheme achieves an accuracy rate of 99.70% in binary classification and 99.41% in multi classification.

Multi-parameter and multi-objective optimization of dual-fuel cell system heavy-duty vehicles: Sizing for serial development

Dual-fuel cell hybrid system provides an attractive propulsion option in transportation, especially for heavy-duty vehicles. However, the larger vehicle weight improves the sensitivity of power demand to road conditions and vehicle handling, making it a challenge to realize reasonable sizing. The scope of this work is to demonstrate a multi-objective and multi-parameter optimization for the serial development of the heavy-duty vehicle, powered by a dual-fuel cell hybrid system. Toward this end, a comprehensive modeling is presented combining the degradation model of the FC system and the battery system. The Pareto theory is introduced to evaluate the three-dimensional objectives involving the equivalent hydrogen consumption, the mass goal, and the vehicle dynamic, which is derived from different six-dimensional parameters under a dual-layer optimization approach. The brute force approach is not applicable in the presence of the curse of dimensionality arising from multi-parameter optimization. The proposed methodology offers a viable approach to acquiring rational sets of sizing solutions in the optimization space with high-dimensional parameters. Considering the serialization of products, the improved solution and the corresponding performance upper limit have be determined according to the proposed methodology under different weight levels as well.

An automated framework for material property calibration in loudspeaker simulation model

The shift to virtual meetings, online classes, and remote work has established a new norm, leading to a surge in the use of virtual communication platforms such as Zoom and Microsoft Teams. This shift has increased the demand for high-quality headsets and speakerphones, emphasizing the need for clear, superior audio quality. The process of calibrating material properties typically relies on repetitive simulations guided by experts' intuition, presenting challenges in establishing new Finite Element Models (FEMs) of loudspeakers, as it requires the repeated identification of material property values. We present a systematic framework for calibrating the mechanical material properties of loudspeaker drivers, a crucial prerequisite for developing accurate FEMs of loudspeakers. Specifically, we propose a statistically-driven approach to replace the conventional manual calibration process, which typically relies on multiple simulations guided by expert intuition. Efficient Global Optimization (EGO) is applied to address the expensive optimization problems of loudspeaker simulation. To tackle the curse of dimensionality, the objective function is decomposed into several functions based on effective parameter groups using Global Sensitivity Analysis (GSA) results. The parameters of the FEM are then calibrated to the reference data from the Lumped Parameter Model (LPM) using the decomposed-reduced objective function, providing the calibrated parameters for the loudspeaker simulation. By implementing this novel approach, even individuals without prior knowledge or experience in loudspeaker material properties can effectively and reliably obtain the necessary data for finite element modeling.

Belief Tsallis-Deng Structure Entropy and its uniform framework for analyzing multivariate time-series complexity based on evidence theory

In the framework of evidence theory, this paper proposes a new measure of uncertainty and introduces an entirely new unified framework for analyzing the complexity of multivariate time series. The theoretical foundation of this paper is random sets, viewing multivariate sequences as a new type of set variable, and using the belief theory framework to describe their behavior. Set variables are a natural extension of point variables, thus providing stronger universality in the analysis of multivariate time series. We tested our proposed new analytical framework on the logistic map and used the Tsallis-Deng Structure Entropy introduced in this paper to analyze the generated multivariate time series. Compared to the traditional Deng entropy, Tsallis-Deng Structure Entropy can adjust the sensitivity to time series complexity by tuning its structural parameter p. As the parameters of the logistic map change, the changes in Tsallis-Deng Structure Entropy show a high correlation with the Lyapunov exponent. In addition, we applied the proposed framework to analyze a multivariate epileptic EEG dataset. The results showed that using the novel evidence theory-based analysis framework for the 23-channel multivariate time series, the average classification accuracies of Deng entropy, Tsallis-Deng entropy, and Tsallis-Deng structural entropy for 24 epilepsy patients were 79 %, 74 %, and 82 %, respectively. In contrast, the average recognition accuracy using the traditional probabilistic framework based on multivariate embedding theory was 64 %. The evidence theory-based framework generally outperformed the multivariate embedding theory, with Tsallis-Deng structural entropy achieving the best performance. These results indicate that the proposed evidence theory-based framework not only avoids the dimensionality curse of the multivariate embedding theory but also better quantifies the complexity of multivariate time series from the same system and distinguishes different types of multivariate time series.

Reinforced steering Evolutionary Markov Chain for high-dimensional feature selection

The increasing accessibility of extensive datasets has amplified the importance of extracting insights from high-dimensional data. However, the task of selecting relevant features in these high-dimensional spaces is made more difficult due to the curse of dimensionality. Although Evolutionary Algorithms (EAs) have shown promise in the literature for feature selection, creating EAs for high dimensions is still challenging. To address the problem of feature selection in high dimensions, a novel concept of Evolutionary Reinforced Markov Chain is proposed in this paper. The proposed work has the following contributions and merits: (i) The paradigms of evolutionary computation, reinforcement learning, and Markov chain are incorporated into an integrational framework for feature selection in high dimensional spaces in a recursive manner. (ii) To support the global convergence of the algorithm and manage its computational complexity, a restricted group of the most effective agents is maintained within the evolutionary population. (iii) The dynamic Markov chain process efficiently manages agent evolution and communication, ensuring effective navigation through the search space. (iv) Agents moving in the right way are rewarded with an increase in their associated transition probability, while the agents going in the wrong direction are discouraged with a decrease in their associated transition probabilities; this promotes the establishment of an equilibrium state and leads to convergence. (v) The effective size of successful agents is reduced recursively while progressing through different states to further facilitate the speed of convergence and decrease the number of features. (vi) The performance comparison with state-of-the-art feature selection methods shows a significant improvement and promise of the proposed method over the existing methods.

Rapid Inverse Parameter Inference Using Physics-Informed Neural Networks

As Li-ion batteries become more essential in today's economy, tools need to be developed to accurately and rapidly diagnose a battery's internal state-of-health. Using a Li-ion battery's (high-rate) voltage response, it is proposed to determine a battery's internal state through Bayesian calibration.However, Bayesian calibration is notoriously slow and requires thousands of model runs. To accelerate parameter inference using Bayesian calibration, a surrogate model is developed to replace the underlying physics-based Li-ion model. Developing a surrogate model for rapid Bayesian calibration analysis is discussed for both the single particle model (SPM) and the pseudo two-dimensional (P2D) model.Surrogate models are constructed using physics-informed neural networks (PINNs) that encode the influence of internal properties on observed voltage responses. In practice, a neural network can be trained by: 1) using simulation results of the physics-based model (i.e., a data-loss approach); 2) using the residuals of the governing equations themselves (i.e., a physics-loss approach); or 3) using a combination of simulation results and governing equation residuals. In the present work, PINNs are developed using a variety of training losses and neural network architectures. In this analysis, it is shown that a PINN surrogate model can be reliably trained with only physics-informed loss. However, using a coupled data-informed and physics-loss approach produced the most accurate PINNs. Figure~\\ref{fig:spm_2d} illustrates the absolute relative errors of trained PINN networks using several different training losses and neural network architectures.After determining a consistent training strategy for both the SPM and P2D PINN surrogate models, the PINNs are extended to determine additional internal state-of-health parameters. As more and more parameters were introduced, the PINN training suffered from ``the curse of dimensionality", which was mitigated by using a hierarchical training approach (where a PINN trained with fewer variable model parameters was used to train a PINN with more variable model parameters).Next, the high-dimensionality PINN surrogates are then integrated into Bayesian calibration schemes to identify internal Li-ion battery properties from experimentally measured voltages. Interpreting the high-dimensional parameter posteriors is discussed with respect to model error, parameter prior choices, and experimental errors. Figure 1

Global Structure Preservation and Self-Representation-Based Supervised Feature Selection

Feature selection aims to select a subset of features from high-dimensional data, which can overcome the curse of dimensionality for the next dealing steps. However, the feature selection itself could face the curse of dimensionality. To overcome the above problem, in this paper, a new feature selection framework is designed according to a human processing in our daily life. In our daily life, to evaluate a candidate's ability to work, the related professional knowledge and the comprehensive ability of a candidate should be both evaluated. Actually, a candidate only with good professional knowledge often hardly solves new problems in the work. Based on the above analysis, in our new designed framework, the features are selected by evaluating its ability of global structure preservation and self-representation, which are respectively similar to the professional knowledge and comprehensive ability in evaluating candidate. As a result, the selected features can accommodate larger changes in test data. The conducted experiments validate the effectiveness of our feature selection.

A meshless stochastic method for Poisson-Nernst-Planck equations.

A plethora of biological, physical, and chemical phenomena involve transport of charged particles (ions). Its continuum-scale description relies on the Poisson-Nernst-Planck (PNP) system, which encapsulates the conservation of mass and charge. The numerical solution of these coupled partial differential equations is challenging and suffers from both the curse of dimensionality and difficulty in efficiently parallelizing. We present a novel particle-based framework to solve the full PNP system by simulating a drift-diffusion process with time- and space-varying drift. We leverage Green's functions, kernel-independent fast multipole methods, and kernel density estimation to solve the PNP system in a meshless manner, capable of handling discontinuous initial states. The method is embarrassingly parallel, and the computational cost scales linearly with the number of particles and dimension. We use a series of numerical experiments to demonstrate both the method's convergence with respect to the number of particles and computational cost vis-à-vis a traditional partial differential equation solver.