High-dimensional Problem Space Research Articles

Transfer learning for high-dimensional linear regression via the elastic net

In this paper, the high-dimensional linear regression problem is explored via the Elastic Net under the transfer learning framework. Within this framework, potentially related source datasets are leveraged to enhance estimation or prediction beyond what can be achieved solely with the target data. When transferable sources are known, an oracle transfer learning algorithm is proposed based on the Elastic Net. Additionally, the ℓ1/ℓ2 estimation error bounds for the corresponding estimator are established. When the transferable sources are unknown, a novel procedure for detecting transferable sources via the Elastic Net is also proposed, with its selection consistency demonstrated under regular conditions. This method transforms the source detection problem into a variable selection problem in high-dimensional space and always gets results that are consistent with the true outcomes. The performance of these methods is further demonstrated through a variety of numerical examples. Finally, our approach is applied to analyze several real datasets for illustrative purposes.

Computation and Applications of Gaussian Integrals in Mathematics and Applied Sciences

Gaussian integrals, particularly those involving the function e−x2 , play a central role in various fields, ranging from physics to finance. This paper explores the computation of Gaussian integrals, beginning with the fundamental integral in the interval −∞ to ∞, which yields π . The author extends this analysis to integrals involving powers of x multiplied by e−x2 , showing their relevance in calculating moments in quantum mechanics and probability theory. Additionally, the author discusses the applications of multivariate Gaussian integrals in machine learning and statistical mechanics, where they are key to solving problems in high-dimensional spaces. Practical examples are provided from quantum mechanics (path integrals), statistical mechanics (partition functions), finance (option pricing models), and machine learning (Gaussian processes). Through these examples, the paper highlights the versatility and universality of Gaussian integrals as essential tools in both theoretical and applied contexts. The integral’s widespread applicability reflects its importance in connecting mathematical theory with real-world phenomena. This work highlights the role played by the Gaussian integral.

Mathematical Techniques for Autonomous Vehicle Navigation Systems

Advanced mathematical methods are used a lot in autonomous car guidance systems to make sure they work correctly, reliably, and efficiently. This abstract talks about some of the most important mathematics methods used to create and improve these systems. One important method is probabilistic robots, which uses Bayesian filters, such as the Kalman filter and its nonlinear versions (Extended Kalman Filter and Unscented Kalman Filter), to estimate a vehicle's state and make sense of sensor data that isn't always clear or loud. Path planning algorithms, like A and Dijkstra's algorithm, are needed to find the best routes. Sampling-based methods, like Rapidly-exploring Random Trees (RRT), can help with problems in high-dimensional space. Control theory is a very important part of keeping a car stable and following the direction you want it to take. Model Predictive Control (MPC), for example, is used a lot because it can handle control jobs with multiple variables while considering the system's behavior. To model how a vehicle moves, differential equations and dynamical systems theory are used to show how control inputs affect how the vehicle acts over time. Also, methods that combine data from different sources like LiDAR, cameras, and GPS are very important for making an accurate and complete picture of the world. Optimization methods improve tracking even more by adjusting the path of the car, cutting down on energy use, and shortening trip times. Besides these methods, machine learning and deep learning are being added to guidance systems more and more to help them make better decisions and be more flexible in settings that are changing quickly and are very complicated. These models can learn from very large datasets by finding trends and making predictions that are important for tasks like finding objects, understanding scenes, and making decisions on their own. Autonomous car tracking systems are getting smarter, more capable, and more reliable by using both old-fashioned math methods and new, cutting-edge machine learning methods together. This makes it possible for them to be widely used in real life.

A short-term wind power forecasting model based on CUR

Abstract Wind power forecasting plays a crucial role in the contemporary renewable energy system. During the process of forecasting wind power, the establishment of LSTM models requires a lot of time and effort, and the interpretability of prediction results is poor, making it difficult to understand and verify the results. To accomplish interpretable and precise wind power predictions, this paper introduces a wind power prediction algorithm model leveraging CUR matrix decomposition. The CUR matrix decomposition method first obtains the original matrix A (wind power data matrix). The statistical influence scores of rows and columns in A are calculated, and several columns and rows with higher scores are selected to form a low dimensional matrix C and R. Matrix C contains the main characteristic factors that affect wind power, while matrix R contains time series features. Then, the matrix U is approximated by A, C, and R to transform the preference feature extraction problem in high-dimensional space into a matrix analysis problem in low-dimensional space, making it more interpretable and accurate. The efficacy of the wind power forecasting approach utilizing CUR matrix decomposition is assessed and confirmed on openly accessible datasets. The results indicate that the CUR matrix decomposition method has good prediction accuracy and interpretability.

A Q-learning driven competitive surrogate assisted evolutionary optimizer with multiple oriented mutation operators for expensive problems

Surrogate-assisted evolutionary algorithms (SAEAs) prevail in the solution of computationally expensive optimization problems. However, with the growth of problem scale and complexity, the high-dimensional problem space greatly restricts their effectiveness and applicability. This paper proposes a Q-learning driven competitive surrogate-assisted evolutionary optimizer with multiple oriented mutation operators (CSEO-MOMO) to improve the prediction effectiveness and accuracy of SAEAs in different scenarios of optimization problems. CSEO-MOMO constructs isomorphic models with different complexities during iteration. The Q-learning method is employed to pick the proper model for prediction based on the state feedback of the iterative population. To explore the search space more comprehensively and increase the probability of finding a better solution, two competitive sub-populations coevolve with personalized mutation operators, of which different oriented optimal tractive base vectors are nominated to guide the mutation direction. In addition, a tailored infilling strategy concerning both the performance of the iterative individuals and their spatial distribution is used to winnow out the individuals with great potential for real evaluation. A suit of high-dimensional CEC benchmarks of different properties and an openly shared topology optimization problem are used for the effectiveness and efficiency verification of CSEO-MOMO. The experimental results reveal that CSEO-MOMO trumps seven excellent SAEAs in solving high-dimensional complex problems and possesses better accuracy and robustness under various fitness landscape scenarios. (The relevant MATLAB code of CSEO-MOMO is publicly available in the first author’s GitHub repository: https://github.com/qinna-zhu/CSEO-MOMO.)

EResNet-SVM: an overfitting-relieved deep learning model for recognition of plant diseases and pests.

The accurate recognition and early warning for plant diseases and pests are a prerequisite of intelligent prevention and control for plant diseases and pests. As a result of the phenotype similarity of the hazarded plant after plant diseases and pests occur, as well as the interference of the external environment, traditional deep learning models often face the overfitting problem in phenotype recognition of plant diseases and pests, which leads to not only the slow convergence speed of the network, but also low recognition accuracy. Motivated by the above problems, the present study proposes a deep learning model EResNet-support vector machine (SVM) to alleviate the overfitting for the recognition and classification of plant diseases and pests. First, the feature extraction capability of the model is improved by increasing feature extraction layers in the convolutional neural network. Second, the order-reduced modules are embedded and a sparsely activated function is introduced to reduce model complexity and alleviate overfitting. Finally, a classifier fused by SVM and fully connected layers are introduced to transforms the original non-linear classification problem into a linear classification problem in high-dimensional space to further alleviate the overfitting and improve the recognition accuracy of plant diseases and pests. The ablation experiments further demonstrate that the fused structure can effectively alleviate the overfitting and improve the recognition accuracy. The experimental recognition results for typical plant diseases and pests show that the proposed EResNet-SVM model has 99.30% test accuracy for eight conditions (seven plant diseases and one normal), which is 5.90% higher than the original ResNet18. Compared with the classic AlexNet, GoogLeNet, Xception, SqueezeNet and DenseNet201 models, the accuracy of the EResNet-SVM model has improved by 5.10%, 7%, 8.10%, 6.20% and 1.90%, respectively. The testing accuracy of the EResNet-SVM model for 6 insect pests is 100%, which is 3.90% higher than that of the original ResNet18 model. This research provides not only useful references for alleviating the overfitting problem in deep learning, but also a theoretical and technical support for the intelligent detection and control of plant diseases and pests. © 2024 Society of Chemical Industry.

A comparison of quaternion neural network backpropagation algorithms

This research paper focuses on quaternion neural networks (QNNs) - a type of neural network wherein the weights, biases, and input values are all represented as quaternion numbers. Previous studies have shown that QNNs outperform real-valued neural networks in basic tasks and have potential in high-dimensional problem spaces. However, research on QNNs has been fragmented, with contributions from different mathematical and engineering domains leading to unintentional overlap in QNN literature. This work aims to unify existing research by evaluating four distinct QNN backpropagation algorithms, including the novel GHR-calculus backpropagation algorithm, and providing concise, scalable implementations of each algorithm using a modern compiled programming language. Additionally, the authors apply a robust Design of Experiments (DoE) methodology to compare the accuracy and runtime of each algorithm. The experiments demonstrate that the Clifford Multilayer Perceptron (CMLP) learning algorithm results in statistically significant improvements in network test set accuracy while maintaining comparable runtime performance to the other three algorithms in four distinct regression tasks. By unifying existing research and comparing different QNN training algorithms, this work develops a state-of-the-art baseline and provides important insights into the potential of QNNs for solving high-dimensional problems.

Cooperative coevolutionary competition swarm optimizer with perturbation for high-dimensional multi-objective optimization

In the realm of high-dimensional problem spaces, particle swarm optimizers have been found to exhibit unnecessary roaming behavior. In response, this paper proposes a cooperative coevolutionary competition swarm optimizer with perturbation (CPCSO) that reduces computational resource consumption. The CPCSO is both simple and effective. Specifically, this optimizer divides the swarm into two sub-swarms, denoted NP1 and NP2. A modified CSO algorithm is used in NP1 to facilitate search space exploration while ensuring that the swarm is well diversified. In NP2, perturbation is introduced to each loser particle to guide it along a smooth granular trajectory, thereby avoiding unnecessary oscillations and improving its capacity to exploit the search space. The two sub-swarms exchange information to balance convergence and distribution, with excellent particles shared between them. Finally, we demonstrate the efficacy of the proposed CPCSO algorithm and several state-of-the-art high-dimensional multi-objective optimizers on the high-dimensional benchmark set LSMOP. Our experimental results indicate that the proposed CPCSO outperforms other algorithms regarding solution quality, convergence speed, and computational cost. Notably, the proposed optimizer demonstrates robust performance across various landscape problems.

Oblique photogrammetry supporting procedural tree modeling in urban areas

Trees are common natural objects widely distributed in urban areas. The modeling quality of trees has significant visual effects for urban modeling. Tree models directly from the oblique photogrammetry pipeline are erratic because tree textures are repeating and confusing. Existing tree modeling approaches either cannot deal with the input data of poor quality or are time-consuming. This paper presents an oblique photogrammetry-supporting procedural tree modeling approach to solve this problem. Specifically, we propose a point cloud data-supported hybrid parametric model that models trees by simulating the growth of natural trees. To solve the challenging optimization problem in high dimensional space defined by the parametric model, a Control Parameter Analysis (CPA)-based optimization method is proposed to find the approximate solution with the highest resemblance to the input data. Finally, an automatic level of detail (LOD) control method of tree models is proposed to form a complete workflow for rendering the urban scenes. Experimental results indicate that the proposed method can generate procedural tree models in urban areas with satisfying accuracy and efficiency. The mean normalized distance between the generated tree models and input data in the test regions is 0.022, and it takes about ten minutes to generate each tree model. Our code is available at https://github.com/lelleMU/Procedrual_Tree_Modeling.

A proposed framework for face - iris recognition system using enhanced mayfly algorithm

Fused biometrics systems have proven to solve some problems associated with unimodal systems but also face challenges in various aspects of their implementation such as difficulty in design, user acceptance is quite low, and the performance tradeoff. This framework tends to address some of these implementation challenges by using an enhanced mayfly algorithm, a modification of the existing mayfly algorithm that was recently proposed, as feature selection. Mayfly algorithm combines advantages of particle swarm optimization, genetic algorithm, and firefly algorithm, simulated in different experiments using varied benchmark function on conventional mayfly algorithm all tested to be capable of optimization, but despite its capabilities, some limitations such as slow convergent or premature convergent rate and possible imbalance between exploration and exploitation still remain unresolved, which necessitated enhancement for better performance. This framework will enhance the existing mayfly algorithm by expanding the search space which limited the ability of the conventional mayfly algorithm to be used to solve high-dimensional problem spaces such as feature selection and modify the selection procedure to model the attraction process as a deterministic process, that will be used for the feature selection procedure on fused face –iris recognition system. This will increase the capabilities of the mayfly algorithm and in turn, increase the recognition accuracy, and reduced the false acceptance rate, false rejection rate, and time complexity of the fused face–iris recognition system.

Behavior segmentation of electricity consumption patterns: A cluster analytical approach

Developing effective energy use and management strategies requires the knowledge of determinants and patterns of the electricity usage behavior of different consumers. This paper examines data-driven unsupervised learning schemes to partition the smart meter users into different clusters such that the users in the same cluster have similar consumption patterns represented by their similar daily routines and peak demand time periods. However, it is critical to resolve the problem of high dimensional space in long time series of smart meter data to make the clustering practical. This problem is resolved by time series data mining based on meaningful statistics and behavioral characteristics to emphasize the essential features for reducing the dimensionality and noisiness of the data. Our work involves rigorous data analysis of smart meter data to identify the essential behavioral characteristics and the source of variability in the behavior and analysis of these behavioral characteristics to select the clustering algorithm and its parameters appropriately. These behavioral features will be extracted from each time-series and then used for clustering using a Gaussian mixture modeling​ algorithm. The clustering algorithm is executed on a real-world electricity consumption dataset of 5038 consumers in Slovenia. The classifications obtained with the algorithm are effective when tested visually and by scoring methods of internal indices. The results indicate that the time of day, day of the week, day-to-day variability and seasonality differences that exist in different clusters can be used to formulate new energy management and grid planning strategies.

Random features for high-dimensional nonlocal mean-field games

We propose an efficient solution approach for high-dimensional nonlocal mean-field game (MFG) systems based on the Monte Carlo approximation of interaction kernels via random features. We avoid costly space-discretizations of interaction terms in the state-space by passing to the feature-space. This approach allows for a seamless mean-field extension of virtually any single-agent trajectory optimization algorithm. Here, we extend the direct transcription approach in optimal control to the mean-field setting. We demonstrate the efficiency of our method by solving MFG problems in high-dimensional spaces which were previously out of reach for conventional non-deep-learning techniques.

A study on the critical Kirchhoff problem in high-dimensional space

A study on the critical Kirchhoff problem in high-dimensional space

Convergence rates of Tikhonov regularizations for elliptic and parabolic inverse radiativity problems

We shall study in this paper the convergence rates of the Tikhonov regularized solutions for the recovery of the radiativities in elliptic and parabolic systems in general dimensional spaces. The conditional stability estimates are first derived. Due to the difficulty of the verification of the existing source conditions or nonlinearity conditions of the inverse radiativity problems in high dimensional spaces, some new variational source conditions are proposed. The conditions are rigorously verified in general dimensional spaces under the conditional stability estimates. We will also derive the reasonable convergence rates under the new source conditions, and the results reveal the explicit relation between the regularity of the radiativities and the convergence rates.

Quantum-inspired algorithm with fitness landscape approximation in reduced dimensional spaces for numerical function optimization

In realistic numerical optimization problems, the efficiency of the algorithm is a crucial consideration. Fitness landscape approximation is beneficial for improving the search efficiency in evolutionary algorithms, but this method increases the computational cost significantly with an increase in the dimension. This paper proposes a quantum-inspired algorithm with dimension decomposition and fitness landscape approximation. The high-dimensional problem space is transformed into a low-dimensional projection space for fitness landscape approximation through mapping transformation. Linear approximation for the fitness landscape is performed in the transformed multiple 1D projected spaces. In accordance with two different diversity-retaining strategies, reference sampling points with different characteristics are selected to perform two-point linear approximation. Candidate points in the projection space are estimated and then synthesized as candidate solutions in the problem space. Then, candidate solutions are used in an appropriate population reconstruction mechanism to improve the performance of the algorithm. The proposed algorithms are implemented on benchmark functions abstracted from real-world optimization problems and compared with several algorithms. Experimental results on errors, success rates, and their rankings show the superiority of the proposed approach with the solution diversity priority with regard to effectiveness, adaptability, and stability.

고차원 볼의 적도 부근의 체적 분포에 대한 정밀 근사

In high-dimensional space, the volume of an n-dimensional ball defined in Euclidean space tends to concentrate near the surface and the equator of the ball. The purpose of this paper is to show that the volume distribution of a high-dimensional ball near the equator can be accurately approximated with a standard normal distribution. Based on the volume formulas of an n-dimensional ball, Stirling’s approximation and binomial approximation are used for a straightforward derivation of the accurate approximation of the ball distribution. Numerical data are given to demonstrate the non-intuitive phenomenon of the volume concentration and the accuracy of the volume distribution approximation. At the end, some discussions are made on the implications of the volume distribution of a high-dimensional ball toward the data search problem in high-dimensional space.

Escaping optimization traps: the role of cultural adaptation and cultural exaptation in facilitating open-ended cumulative dynamics

Explaining the origins of cumulative culture, and how it is maintained over long timescales, constitutes a challenge for theories of cultural evolution. Previous theoretical work has emphasized two fundamental causal processes: cultural adaptation (where technologies are refined towards a functional objective) and cultural exaptation (the repurposing of existing technologies towards a new functional goal). Yet, despite the prominence of cultural exaptation in theoretical explanations, this process is often absent from models and experiments of cumulative culture. Using an agent-based model, where agents attempt to solve problems in a high-dimensional problem space, the current paper investigates the relationship between cultural adaptation and cultural exaptation and produces three major findings. First, cultural dynamics often end up in optimization traps: here, the process of optimization causes the dynamics of change to cease, with populations entering a state of equilibrium. Second, escaping these optimization traps requires cultural dynamics to explore the problem space rapidly enough to create a moving target for optimization. This results in a positive feedback loop of open-ended growth in both the diversity and complexity of cultural solutions. Finally, the results helped delineate the roles played by social and asocial mechanisms: asocial mechanisms of innovation drive the emergence of cumulative culture and social mechanisms of within-group transmission help maintain these dynamics over long timescales.

LSH-based distributed similarity indexing with load balancing in high-dimensional space

Locality-sensitive hashing (LSH) and its variants are well-known indexing schemes for solving the similarity search problem in high-dimensional space. Traditionally, these indexing schemes are centrally managed and multiple hash tables are needed to guarantee the search quality. However, due to the limitation of storage space and processing capacity of the server, the centralized indexing schemes become impractical for massive data objects. Therefore, several distributed indexing schemes based on peer-to-peer (P2P) networks are proposed, whereas how to ensure load balancing is still one of the key issues. To solve the problem, in this paper, we propose two theoretical LSH-based data distribution models in P2P networks for datasets with homogeneous and heterogeneous $$l_2$$ norms, respectively. Unlike earlier schemes, to our knowledge, we focus on load balancing for a single hash table rather than multiple tables, which has not been considered previously. Then, we propose a static distributed indexing scheme with a novel load balancing indexing mapping method based on the cumulative distribution function by our models. Furthermore, we propose a dynamic load rebalancing algorithm using virtual node method of P2P networks to make the static indexing scheme more practical and robust. The experiments based on synthetic and real datasets show that the proposed distributed similarity indexing schemes are effective and efficient for load balancing in similarity indexing of high-dimensional space.

A prediction model for population change using ARIMA model based on feature extraction

Population is a social entity with complex contents and a variety of social relations. It includes gender, age and natural composition, as well as a variety of social composition, social relations, economic composition and economic relations. It is difficult to analyze the population change in large number and high dimension over the years in China. A prediction model for population change using ARIMA model based on feature extraction is proposed. It provides the reduction of population change indicators and predicts the population change in the future. Principal component analysis is firstly used to transform this problem in high-dimensional space into low-dimensional space, making the problem simple and intuitive. These less comprehensive indexes obtained are not related to each other and provide most information of the original indexes. The data of the national population change from 1979 to 2015 is effectively reduced to one principal component. On the basis of the analysis results, a prediction model using ARIMA model is established to predict the population change in the next few years.

Movement patterns of a particle swarm in high dimensional spaces

In high dimensional problem spaces, particle swarm optimization (PSO) is prone to unwanted roaming behaviour due to initial velocity explosion. A particle swarm’s movement patterns are strongly influenced by the inertia weight and acceleration coefficients. This paper investigates whether the initial velocity explosion can be curbed by appropriate choice of the inertia weight and the acceleration coefficients, which restrict the standard deviation of particle positions. It is shown that roaming behaviour cannot be solved by reducing swarm variance directly, but that the relationship between the parameters must also be considered. Furthermore, the paper investigates different movement patterns that may be exhibited by the swarm. It is shown that optimal parameter configurations differ between low and high dimensional problems. Specifically, parameter configurations which produce very smooth particle trajectories and restrict the swarm’s movement range are advantageous in high dimensional spaces. These movement patterns correspond to high inertia weight and low acceleration coefficients (eg. w=0.9694,c1=c2=0.099381). Swarms with smooth particle trajectories exhibited significantly less unwanted roaming behaviour than swarms with chaotic or oscillating particle trajectories.