High-dimensional Problem Space Research Articles

EResNet-SVM: an overfitting-relieved deep learning model for recognition of plant diseases and pests.

The accurate recognition and early warning for plant diseases and pests are a prerequisite of intelligent prevention and control for plant diseases and pests. As a result of the phenotype similarity of the hazarded plant after plant diseases and pests occur, as well as the interference of the external environment, traditional deep learning models often face the overfitting problem in phenotype recognition of plant diseases and pests, which leads to not only the slow convergence speed of the network, but also low recognition accuracy. Motivated by the above problems, the present study proposes a deep learning model EResNet-support vector machine (SVM) to alleviate the overfitting for the recognition and classification of plant diseases and pests. First, the feature extraction capability of the model is improved by increasing feature extraction layers in the convolutional neural network. Second, the order-reduced modules are embedded and a sparsely activated function is introduced to reduce model complexity and alleviate overfitting. Finally, a classifier fused by SVM and fully connected layers are introduced to transforms the original non-linear classification problem into a linear classification problem in high-dimensional space to further alleviate the overfitting and improve the recognition accuracy of plant diseases and pests. The ablation experiments further demonstrate that the fused structure can effectively alleviate the overfitting and improve the recognition accuracy. The experimental recognition results for typical plant diseases and pests show that the proposed EResNet-SVM model has 99.30% test accuracy for eight conditions (seven plant diseases and one normal), which is 5.90% higher than the original ResNet18. Compared with the classic AlexNet, GoogLeNet, Xception, SqueezeNet and DenseNet201 models, the accuracy of the EResNet-SVM model has improved by 5.10%, 7%, 8.10%, 6.20% and 1.90%, respectively. The testing accuracy of the EResNet-SVM model for 6 insect pests is 100%, which is 3.90% higher than that of the original ResNet18 model. This research provides not only useful references for alleviating the overfitting problem in deep learning, but also a theoretical and technical support for the intelligent detection and control of plant diseases and pests. © 2024 Society of Chemical Industry.

A comparison of quaternion neural network backpropagation algorithms

This research paper focuses on quaternion neural networks (QNNs) - a type of neural network wherein the weights, biases, and input values are all represented as quaternion numbers. Previous studies have shown that QNNs outperform real-valued neural networks in basic tasks and have potential in high-dimensional problem spaces. However, research on QNNs has been fragmented, with contributions from different mathematical and engineering domains leading to unintentional overlap in QNN literature. This work aims to unify existing research by evaluating four distinct QNN backpropagation algorithms, including the novel GHR-calculus backpropagation algorithm, and providing concise, scalable implementations of each algorithm using a modern compiled programming language. Additionally, the authors apply a robust Design of Experiments (DoE) methodology to compare the accuracy and runtime of each algorithm. The experiments demonstrate that the Clifford Multilayer Perceptron (CMLP) learning algorithm results in statistically significant improvements in network test set accuracy while maintaining comparable runtime performance to the other three algorithms in four distinct regression tasks. By unifying existing research and comparing different QNN training algorithms, this work develops a state-of-the-art baseline and provides important insights into the potential of QNNs for solving high-dimensional problems.

Cooperative coevolutionary competition swarm optimizer with perturbation for high-dimensional multi-objective optimization

In the realm of high-dimensional problem spaces, particle swarm optimizers have been found to exhibit unnecessary roaming behavior. In response, this paper proposes a cooperative coevolutionary competition swarm optimizer with perturbation (CPCSO) that reduces computational resource consumption. The CPCSO is both simple and effective. Specifically, this optimizer divides the swarm into two sub-swarms, denoted NP1 and NP2. A modified CSO algorithm is used in NP1 to facilitate search space exploration while ensuring that the swarm is well diversified. In NP2, perturbation is introduced to each loser particle to guide it along a smooth granular trajectory, thereby avoiding unnecessary oscillations and improving its capacity to exploit the search space. The two sub-swarms exchange information to balance convergence and distribution, with excellent particles shared between them. Finally, we demonstrate the efficacy of the proposed CPCSO algorithm and several state-of-the-art high-dimensional multi-objective optimizers on the high-dimensional benchmark set LSMOP. Our experimental results indicate that the proposed CPCSO outperforms other algorithms regarding solution quality, convergence speed, and computational cost. Notably, the proposed optimizer demonstrates robust performance across various landscape problems.

Oblique photogrammetry supporting procedural tree modeling in urban areas

Trees are common natural objects widely distributed in urban areas. The modeling quality of trees has significant visual effects for urban modeling. Tree models directly from the oblique photogrammetry pipeline are erratic because tree textures are repeating and confusing. Existing tree modeling approaches either cannot deal with the input data of poor quality or are time-consuming. This paper presents an oblique photogrammetry-supporting procedural tree modeling approach to solve this problem. Specifically, we propose a point cloud data-supported hybrid parametric model that models trees by simulating the growth of natural trees. To solve the challenging optimization problem in high dimensional space defined by the parametric model, a Control Parameter Analysis (CPA)-based optimization method is proposed to find the approximate solution with the highest resemblance to the input data. Finally, an automatic level of detail (LOD) control method of tree models is proposed to form a complete workflow for rendering the urban scenes. Experimental results indicate that the proposed method can generate procedural tree models in urban areas with satisfying accuracy and efficiency. The mean normalized distance between the generated tree models and input data in the test regions is 0.022, and it takes about ten minutes to generate each tree model. Our code is available at https://github.com/lelleMU/Procedrual_Tree_Modeling.

A proposed framework for face - iris recognition system using enhanced mayfly algorithm

Fused biometrics systems have proven to solve some problems associated with unimodal systems but also face challenges in various aspects of their implementation such as difficulty in design, user acceptance is quite low, and the performance tradeoff. This framework tends to address some of these implementation challenges by using an enhanced mayfly algorithm, a modification of the existing mayfly algorithm that was recently proposed, as feature selection. Mayfly algorithm combines advantages of particle swarm optimization, genetic algorithm, and firefly algorithm, simulated in different experiments using varied benchmark function on conventional mayfly algorithm all tested to be capable of optimization, but despite its capabilities, some limitations such as slow convergent or premature convergent rate and possible imbalance between exploration and exploitation still remain unresolved, which necessitated enhancement for better performance. This framework will enhance the existing mayfly algorithm by expanding the search space which limited the ability of the conventional mayfly algorithm to be used to solve high-dimensional problem spaces such as feature selection and modify the selection procedure to model the attraction process as a deterministic process, that will be used for the feature selection procedure on fused face –iris recognition system. This will increase the capabilities of the mayfly algorithm and in turn, increase the recognition accuracy, and reduced the false acceptance rate, false rejection rate, and time complexity of the fused face–iris recognition system.

Behavior segmentation of electricity consumption patterns: A cluster analytical approach

Developing effective energy use and management strategies requires the knowledge of determinants and patterns of the electricity usage behavior of different consumers. This paper examines data-driven unsupervised learning schemes to partition the smart meter users into different clusters such that the users in the same cluster have similar consumption patterns represented by their similar daily routines and peak demand time periods. However, it is critical to resolve the problem of high dimensional space in long time series of smart meter data to make the clustering practical. This problem is resolved by time series data mining based on meaningful statistics and behavioral characteristics to emphasize the essential features for reducing the dimensionality and noisiness of the data. Our work involves rigorous data analysis of smart meter data to identify the essential behavioral characteristics and the source of variability in the behavior and analysis of these behavioral characteristics to select the clustering algorithm and its parameters appropriately. These behavioral features will be extracted from each time-series and then used for clustering using a Gaussian mixture modeling​ algorithm. The clustering algorithm is executed on a real-world electricity consumption dataset of 5038 consumers in Slovenia. The classifications obtained with the algorithm are effective when tested visually and by scoring methods of internal indices. The results indicate that the time of day, day of the week, day-to-day variability and seasonality differences that exist in different clusters can be used to formulate new energy management and grid planning strategies.

Random features for high-dimensional nonlocal mean-field games

We propose an efficient solution approach for high-dimensional nonlocal mean-field game (MFG) systems based on the Monte Carlo approximation of interaction kernels via random features. We avoid costly space-discretizations of interaction terms in the state-space by passing to the feature-space. This approach allows for a seamless mean-field extension of virtually any single-agent trajectory optimization algorithm. Here, we extend the direct transcription approach in optimal control to the mean-field setting. We demonstrate the efficiency of our method by solving MFG problems in high-dimensional spaces which were previously out of reach for conventional non-deep-learning techniques.

A study on the critical Kirchhoff problem in high-dimensional space

A study on the critical Kirchhoff problem in high-dimensional space

Convergence rates of Tikhonov regularizations for elliptic and parabolic inverse radiativity problems

We shall study in this paper the convergence rates of the Tikhonov regularized solutions for the recovery of the radiativities in elliptic and parabolic systems in general dimensional spaces. The conditional stability estimates are first derived. Due to the difficulty of the verification of the existing source conditions or nonlinearity conditions of the inverse radiativity problems in high dimensional spaces, some new variational source conditions are proposed. The conditions are rigorously verified in general dimensional spaces under the conditional stability estimates. We will also derive the reasonable convergence rates under the new source conditions, and the results reveal the explicit relation between the regularity of the radiativities and the convergence rates.

Quantum-inspired algorithm with fitness landscape approximation in reduced dimensional spaces for numerical function optimization

In realistic numerical optimization problems, the efficiency of the algorithm is a crucial consideration. Fitness landscape approximation is beneficial for improving the search efficiency in evolutionary algorithms, but this method increases the computational cost significantly with an increase in the dimension. This paper proposes a quantum-inspired algorithm with dimension decomposition and fitness landscape approximation. The high-dimensional problem space is transformed into a low-dimensional projection space for fitness landscape approximation through mapping transformation. Linear approximation for the fitness landscape is performed in the transformed multiple 1D projected spaces. In accordance with two different diversity-retaining strategies, reference sampling points with different characteristics are selected to perform two-point linear approximation. Candidate points in the projection space are estimated and then synthesized as candidate solutions in the problem space. Then, candidate solutions are used in an appropriate population reconstruction mechanism to improve the performance of the algorithm. The proposed algorithms are implemented on benchmark functions abstracted from real-world optimization problems and compared with several algorithms. Experimental results on errors, success rates, and their rankings show the superiority of the proposed approach with the solution diversity priority with regard to effectiveness, adaptability, and stability.

고차원 볼의 적도 부근의 체적 분포에 대한 정밀 근사

In high-dimensional space, the volume of an n-dimensional ball defined in Euclidean space tends to concentrate near the surface and the equator of the ball. The purpose of this paper is to show that the volume distribution of a high-dimensional ball near the equator can be accurately approximated with a standard normal distribution. Based on the volume formulas of an n-dimensional ball, Stirling’s approximation and binomial approximation are used for a straightforward derivation of the accurate approximation of the ball distribution. Numerical data are given to demonstrate the non-intuitive phenomenon of the volume concentration and the accuracy of the volume distribution approximation. At the end, some discussions are made on the implications of the volume distribution of a high-dimensional ball toward the data search problem in high-dimensional space.

Escaping optimization traps: the role of cultural adaptation and cultural exaptation in facilitating open-ended cumulative dynamics

Explaining the origins of cumulative culture, and how it is maintained over long timescales, constitutes a challenge for theories of cultural evolution. Previous theoretical work has emphasized two fundamental causal processes: cultural adaptation (where technologies are refined towards a functional objective) and cultural exaptation (the repurposing of existing technologies towards a new functional goal). Yet, despite the prominence of cultural exaptation in theoretical explanations, this process is often absent from models and experiments of cumulative culture. Using an agent-based model, where agents attempt to solve problems in a high-dimensional problem space, the current paper investigates the relationship between cultural adaptation and cultural exaptation and produces three major findings. First, cultural dynamics often end up in optimization traps: here, the process of optimization causes the dynamics of change to cease, with populations entering a state of equilibrium. Second, escaping these optimization traps requires cultural dynamics to explore the problem space rapidly enough to create a moving target for optimization. This results in a positive feedback loop of open-ended growth in both the diversity and complexity of cultural solutions. Finally, the results helped delineate the roles played by social and asocial mechanisms: asocial mechanisms of innovation drive the emergence of cumulative culture and social mechanisms of within-group transmission help maintain these dynamics over long timescales.

LSH-based distributed similarity indexing with load balancing in high-dimensional space

Locality-sensitive hashing (LSH) and its variants are well-known indexing schemes for solving the similarity search problem in high-dimensional space. Traditionally, these indexing schemes are centrally managed and multiple hash tables are needed to guarantee the search quality. However, due to the limitation of storage space and processing capacity of the server, the centralized indexing schemes become impractical for massive data objects. Therefore, several distributed indexing schemes based on peer-to-peer (P2P) networks are proposed, whereas how to ensure load balancing is still one of the key issues. To solve the problem, in this paper, we propose two theoretical LSH-based data distribution models in P2P networks for datasets with homogeneous and heterogeneous $$l_2$$ norms, respectively. Unlike earlier schemes, to our knowledge, we focus on load balancing for a single hash table rather than multiple tables, which has not been considered previously. Then, we propose a static distributed indexing scheme with a novel load balancing indexing mapping method based on the cumulative distribution function by our models. Furthermore, we propose a dynamic load rebalancing algorithm using virtual node method of P2P networks to make the static indexing scheme more practical and robust. The experiments based on synthetic and real datasets show that the proposed distributed similarity indexing schemes are effective and efficient for load balancing in similarity indexing of high-dimensional space.

A prediction model for population change using ARIMA model based on feature extraction

Population is a social entity with complex contents and a variety of social relations. It includes gender, age and natural composition, as well as a variety of social composition, social relations, economic composition and economic relations. It is difficult to analyze the population change in large number and high dimension over the years in China. A prediction model for population change using ARIMA model based on feature extraction is proposed. It provides the reduction of population change indicators and predicts the population change in the future. Principal component analysis is firstly used to transform this problem in high-dimensional space into low-dimensional space, making the problem simple and intuitive. These less comprehensive indexes obtained are not related to each other and provide most information of the original indexes. The data of the national population change from 1979 to 2015 is effectively reduced to one principal component. On the basis of the analysis results, a prediction model using ARIMA model is established to predict the population change in the next few years.

Movement patterns of a particle swarm in high dimensional spaces

In high dimensional problem spaces, particle swarm optimization (PSO) is prone to unwanted roaming behaviour due to initial velocity explosion. A particle swarm’s movement patterns are strongly influenced by the inertia weight and acceleration coefficients. This paper investigates whether the initial velocity explosion can be curbed by appropriate choice of the inertia weight and the acceleration coefficients, which restrict the standard deviation of particle positions. It is shown that roaming behaviour cannot be solved by reducing swarm variance directly, but that the relationship between the parameters must also be considered. Furthermore, the paper investigates different movement patterns that may be exhibited by the swarm. It is shown that optimal parameter configurations differ between low and high dimensional problems. Specifically, parameter configurations which produce very smooth particle trajectories and restrict the swarm’s movement range are advantageous in high dimensional spaces. These movement patterns correspond to high inertia weight and low acceleration coefficients (eg. w=0.9694,c1=c2=0.099381). Swarms with smooth particle trajectories exhibited significantly less unwanted roaming behaviour than swarms with chaotic or oscillating particle trajectories.

Differentially Private Empirical Risk Minimization with Smooth Non-Convex Loss Functions: A Non-Stationary View

In this paper, we study the Differentially Private Empirical Risk Minimization (DP-ERM) problem with non-convex loss functions and give several upper bounds for the utility in different settings. We first consider the problem in low-dimensional space. For DP-ERM with non-smooth regularizer, we generalize an existing work by measuring the utility using ℓ2 norm of the projected gradient. Also, we extend the error bound measurement, for the first time, from empirical risk to population risk by using the expected ℓ2 norm of the gradient. We then investigate the problem in high dimensional space, and show that by measuring the utility with Frank-Wolfe gap, it is possible to bound the utility by the Gaussian Width of the constraint set, instead of the dimensionality p of the underlying space. We further demonstrate that the advantages of this result can be achieved by the measure of ℓ2 norm of the projected gradient. A somewhat surprising discovery is that although the two kinds of measurements are quite different, their induced utility upper bounds are asymptotically the same under some assumptions. We also show that the utility of some special non-convex loss functions can be reduced to a level (i.e., depending only on log p) similar to that of convex loss functions. Finally, we test our proposed algorithms on both synthetic and real world datasets and the experimental results confirm our theoretical analysis.

Robot Motion Planning Method Based on Incremental High-Dimensional Mixture Probabilistic Model

The sampling-based motion planner is the mainstream method to solve the motion planning problem in high-dimensional space. In the process of exploring robot configuration space, this type of algorithm needs to perform collision query on a large number of samples, which greatly limits their planning efficiency. Therefore, this paper uses machine learning methods to establish a probabilistic model of the obstacle region in configuration space by learning a large number of labeled samples. Based on this, the high-dimensional samples’ rapid collision query is realized. The influence of number of Gaussian components on the fitting accuracy is analyzed in detail, and a self-adaptive model training method based on Greedy expectation-maximization (EM) algorithm is proposed. At the same time, this method has the capability of online updating and can eliminate model fitting errors due to environmental changes. Finally, the model is combined with a variety of sampling-based motion planners and is validated in multiple sets of simulations and real world experiments. The results show that, compared with traditional methods, the proposed method has significantly improved the planning efficiency.

An Active Learning Classifier for Further Reducing Diabetic Retinopathy Screening System Cost.

Diabetic retinopathy (DR) screening system raises a financial problem. For further reducing DR screening cost, an active learning classifier is proposed in this paper. Our approach identifies retinal images based on features extracted by anatomical part recognition and lesion detection algorithms. Kernel extreme learning machine (KELM) is a rapid classifier for solving classification problems in high dimensional space. Both active learning and ensemble technique elevate performance of KELM when using small training dataset. The committee only proposes necessary manual work to doctor for saving cost. On the publicly available Messidor database, our classifier is trained with 20%–35% of labeled retinal images and comparative classifiers are trained with 80% of labeled retinal images. Results show that our classifier can achieve better classification accuracy than Classification and Regression Tree, radial basis function SVM, Multilayer Perceptron SVM, Linear SVM, and K Nearest Neighbor. Empirical experiments suggest that our active learning classifier is efficient for further reducing DR screening cost.

A Chaos-Enhanced Particle Swarm Optimization with Adaptive Parameters and Its Application in Maximum Power Point Tracking

This work proposes an enhanced particle swarm optimization scheme that improves upon the performance of the standard particle swarm optimization algorithm. The proposed algorithm is based on chaos search to solve the problems of stagnation, which is the problem of being trapped in a local optimum and with the risk of premature convergence. Type1′′constriction is incorporated to help strengthen the stability and quality of convergence, and adaptive learning coefficients are utilized to intensify the exploitation and exploration search characteristics of the algorithm. Several well known benchmark functions are operated to verify the effectiveness of the proposed method. The test performance of the proposed method is compared with those of other popular population-based algorithms in the literature. Simulation results clearly demonstrate that the proposed method exhibits faster convergence, escapes local minima, and avoids premature convergence and stagnation in a high-dimensional problem space. The validity of the proposed PSO algorithm is demonstrated using a fuzzy logic-based maximum power point tracking control model for a standalone solar photovoltaic system.

Application of comprehensive learning particle swarm optimisation algorithm for maximum likelihood DOA estimation in wireless sensor networks

Direction of arrival (DOA) estimation is one of the challenging problem in wireless sensor networks. Several methods based on maximum likelihood (ML) criteria have been established in literature. But, the multimodal nature of ML cost function is one of the inherent limitations in ML-DOA estimation technique. Generally, to obtain the likelihood solutions, the DOAs must be estimated by optimising a complicated function over a high-dimensional problem space. Recently particle swarm optimisation (PSO) algorithm is used in MLDOA estimation. To overcome the drawback of premature convergence in PSO, a learning strategy is introduced, and this approach called comprehensive learning particle swarm optimisation (CLPSO), which is applied to this problem and a comparison of results is made between these two. Simulation results confirms that the ML-CLPSO estimator is significantly giving better performance at low signal-to-noise ratio compared to conventional methods like multiple signal classification (MUSIC) and ML-PSO in various scenarios at less computational costs.