High-dimensional Optimization Problems Research Articles

A hybrid biogeography-based optimization algorithm to solve high-dimensional optimization problems and real-world engineering problems

According to our extensive investigation, Biogeography-based optimization (BBO) and its variants have not been applied to solve high-dimensional optimization problems. To make a breakthrough in this field, a new BBO variant with hybrid migration operator and feedback differential evolution mechanism, HFBBO, is proposed. Firstly, the example learning method is used to ensure the inferior solutions cannot destroy the superior solutions. Secondly, the hybrid migration operator is presented to balance the exploration and exploitation. It enables the algorithm to switch freely between local search and global search. Finally, the feedback differential evolution mechanism is designed to replace the random mutation operator. HFBBO can select the mutation mode intelligently by this mechanism to avoid getting stuck in local optima. Meanwhile, the Markov model is established to prove the convergence of HFBBO, and the complexity is also discussed. A series experiments are carried out on 24 benchmark functions, CEC2017 test suite and 12 real-world engineering problems. The results of the Wilcoxon’s rank-sum test and Friedman’s test show that HFBBO has better competitiveness and stability than the 27 compared algorithms. Furtherly, the performance of HFBBO is compared on 1000, 2000, 5000 and 10000 dimensions, respectively. Experimental results show that this method can effectively solve high-dimensional optimization problems.

Random following ant colony optimization: Continuous and binary variants for global optimization and feature selection

Continuous ant colony optimization was a population-based heuristic search algorithm inspired by the pathfinding behavior of ant colonies with a simple structure and few control parameters. However, in the case of multimodal and high-dimensional optimization problems, it was often limited to local regions in the feasible domain space, negatively affecting the computational effort required to find the optimal solution point. To alleviate its limitations in this regard, a random following strategy is proposed to enhance communication among the ant colony search agent and other ant colony members within the search dimension. The proposed algorithm that incorporates this strategy is called Random Following Ant Colony Optimization. Then, to evaluate the global optimization performance of the proposed algorithm, the well-known numerical optimization problem, namely the Congress on Evolutionary Computation 2017 test suite, is used. First, the proposed algorithm’s parameters are analyzed for sensitivity, scalability experiments, and balanced diversity. Second, it is compared experimentally with 11 state-of-the-art algorithms in dimensions 10, 30, 50, and 100, respectively, and Wilcoxon signed-rank test, Friedman test, and Bonferroni-Dunn post-hoc statistical test are used to synthesize the experimental comparison results. Finally, to evaluate the ability of the proposed algorithm to handle discrete feature selection problems, comparative experiments are conducted on 24 datasets with eight well-known classification methods and five high-performance classification methods. The benchmark test results show that the global optimization performance of the proposed algorithm is comparable to the winners of the test suite in 50 and 100 dimensions. The results of the feature selection experiments show that the proposed algorithm is much stronger than the well-known and high-performance classification methods on high-dimensional datasets.

An Adaptive Dual-Population Collaborative Chicken Swarm Optimization Algorithm for High-Dimensional Optimization.

With the development of science and technology, many optimization problems in real life have developed into high-dimensional optimization problems. The meta-heuristic optimization algorithm is regarded as an effective method to solve high-dimensional optimization problems. However, considering that traditional meta-heuristic optimization algorithms generally have problems such as low solution accuracy and slow convergence speed when solving high-dimensional optimization problems, an adaptive dual-population collaborative chicken swarm optimization (ADPCCSO) algorithm is proposed in this paper, which provides a new idea for solving high-dimensional optimization problems. First, in order to balance the algorithm's search abilities in terms of breadth and depth, the value of parameter G is given by an adaptive dynamic adjustment method. Second, in this paper, a foraging-behavior-improvement strategy is utilized to improve the algorithm's solution accuracy and depth-optimization ability. Third, the artificial fish swarm algorithm (AFSA) is introduced to construct a dual-population collaborative optimization strategy based on chicken swarms and artificial fish swarms, so as to improve the algorithm's ability to jump out of local extrema. The simulation experiments on the 17 benchmark functions preliminarily show that the ADPCCSO algorithm is superior to some swarm-intelligence algorithms such as the artificial fish swarm algorithm (AFSA), the artificial bee colony (ABC) algorithm, and the particle swarm optimization (PSO) algorithm in terms of solution accuracy and convergence performance. In addition, the APDCCSO algorithm is also utilized in the parameter estimation problem of the Richards model to further verify its performance.

Combining deep learning approaches and point spread function engineering for simultaneous 3D position and 3D orientation measurements of fluorescent single molecules

Point Spread Function (PSF) engineering is an effective method to increase the sensitivity of single-molecule fluorescence images to specific parameters. Classical phase mask optimization approaches have enabled the creation of new PSFs that can achieve, for example, localization precision of a few nanometers axially over a capture range of several microns with bright emitters. However, for complex high-dimensional optimization problems, classical approaches are difficult to implement and can be very time-consuming for computation. The advent of deep learning methods and their application to single-molecule imaging has provided a way to solve these problems. Here, we propose to combine PSF engineering and deep learning approaches to obtain both an optimized phase mask and a neural network structure to obtain the 3D position and 3D orientation of fixed fluorescent molecules. Our approach allows us to obtain an axial localization precision around 30 nanometers, as well as an orientation precision around 5 degrees for orientations and positions over a one micron depth range for a signal-to-noise ratio consistent with what is typical in single-molecule cellular imaging experiments.

Reduced order model predictive control for parametrized parabolic partial differential equations

Model Predictive Control (MPC) is a well-established approach to solve infinite horizon optimal control problems. Since optimization over an infinite time horizon is generally infeasible, MPC determines a suboptimal feedback control by repeatedly solving finite time optimal control problems. Although MPC has been successfully used in many applications, applying MPC to large-scale systems – arising, e.g., through discretization of partial differential equations – requires the solution of high-dimensional optimization problems and thus poses immense computational effort. We consider systems governed by parametrized parabolic partial differential equations and employ the reduced basis method as a low-dimensional surrogate model for the finite time optimal control problem. The reduced order optimal control serves as feedback control for the original large-scale system. We analyze the proposed RB-MPC approach by first developing a posteriori error bounds for the errors in the optimal control and associated cost functional. These bounds can be evaluated efficiently in an offline-online computational procedure and allow us to guarantee asymptotic stability of the closed-loop system using the RB-MPC approach in several practical scenarios. We also propose an adaptive strategy to choose the prediction horizon of the finite time optimal control problem. Numerical results are presented to illustrate the theoretical properties of our approach.

Heterogeneous Transformer: A Scale Adaptable Neural Network Architecture for Device Activity Detection

To support modern machine-type communications, a crucial task during the random access phase is device activity detection, which is to identify the active devices from a large number of potential devices based on the received signal at the access point. By utilizing the statistical properties of the channel, state-of-the-art covariance based methods have been demonstrated to achieve better activity detection performance than compressed sensing based methods. However, covariance based methods require to solve a high dimensional nonconvex optimization problem by updating the estimate of the activity status of each device sequentially. Since the number of updates is proportional to the device number, the computational complexity and delay make the iterative updates difficult for real-time implementation especially when the device number scales up. Inspired by the success of deep learning for real-time inference, this paper proposes a learning based method with a customized heterogeneous transformer architecture for device activity detection. By adopting an attention mechanism in the architecture design, the proposed method is able to extract features reflecting relevance among device pilots and received signal, permutation equivariant with respect to devices, and its training parameter number is independent of the device number. Simulation results demonstrate that the proposed method achieves better activity detection performance with much shorter computation time than state-of-the-art covariance approach, and generalizes well to different numbers of devices and BS-antennas, different pilot lengths, transmit powers, and cell radii.

Design of Robust Polynomial Beamformers Using Worst Case Performance Optimization via Alternating Direction Method of Multipliers

This paper studies the design of robust polynomial beamformers (RPBs) for microphone arrays, which enable dynamic beam steering via simple online parameter tuning, under the criterion of worst-case performance optimization (WCPO). Although the WCPO criterion has been widely adopted in the design of various robust beamformers, conventional approaches using the WCPO criterion may not be applicable to the RPB design, due to the conservativeness problem resulting from the special structure of polynomial beamformers. In this paper, we propose to design the RPBs based on the semidefinite programming (SDP) using the WCPO criterion. Nevertheless, the resulting design optimization problems are highly computationally demanding. To overcome the computational difficulty, we then propose to solve the high-dimensional optimization problems for the RPB design using the alternating direction method of multipliers (ADMM) algorithm. By introducing auxiliary variables in the framework of ADMM, the original high-dimensional optimization problems are decomposed into several subproblems which are easier to handle. The algorithms to solve the corresponding subproblems are developed. Simulation and real-world experimental results show that the proposed SDP-based RPB design performs much better than its existing counterpart. Moreover, the computational efficiency can be significantly improved by using the proposed ADMM algorithm while the existing solution is very time-consuming and even fails.

AutoBCS: Block-Based Image Compressive Sensing With Data-Driven Acquisition and Noniterative Reconstruction.

Block compressive sensing (CS) is a well-known signal acquisition and reconstruction paradigm with widespread application prospects in science, engineering, and cybernetic systems. However, state-of-the-art block-based image CS (BCS) methods generally suffer from two issues. The sparsifying domain and the sensing matrices widely used for image acquisition are not data driven and, thus, both the features of the image and the relationships among subblock images are ignored. Moreover, it requires to address a high-dimensional optimization problem with extensive computational complexity for image reconstruction. In this article, we provide a deep learning (DL) strategy for BCS, called AutoBCS, which automatically takes the prior knowledge of images into account in the acquisition step and establishes a reconstruction model for performing fast image reconstruction. More precisely, we present a learning-based sensing matrix to accomplish image acquisition, thereby capturing and preserving more image characteristics than those captured by the existing methods. In addition, we build a noniterative reconstruction network, which provides an end-to-end BCS reconstruction framework to maximize image reconstruction efficiency. Furthermore, we investigate comprehensive comparison studies with both traditional BCS approaches and newly developed DL methods. Compared with these approaches, our proposed AutoBCS can not only provide superior performance in terms of image quality metrics (SSIM and PSNR) and visual perception but also automatically benefit reconstruction speed.

Data-driven nonparametric robust control under dependence uncertainty

We consider a multi-period stochastic control problem where the multivariate driving stochastic factor of the system has known marginal distributions but uncertain dependence structure. To solve the problem, we propose to implement the nonparametric adaptive robust control framework. We aim to find the optimal control against the worst-case copulae in a sequence of shrinking uncertainty sets which are generated from continuously observing the data. Then, we use a stochastic gradient descent ascent algorithm to numerically handle the corresponding high dimensional dynamic inf-sup optimization problem. We present the numerical results in the context of utility maximization and show that the controller benefits from knowing more information about the uncertain model.

Computation of the self-diffusion coefficient with low-rank tensor methods: application to the simulation of a cross-diffusion system

Cross-diffusion systems arise as hydrodynamic limits of lattice multi-species interacting particle models. The objective of this work is to provide a numerical scheme for the simulation of the cross-diffusion system identified in [J. Quastel, Comm. Pure Appl. Math., 45 (1992), pp. 623–679]. To simulate this system, it is necessary to provide an approximation of the so-called self-diffusion coefficient matrix of the tagged particle process. Classical algorithms for the computation of this matrix are based on the estimation of the long-time limit of the average mean square displacement of the particle. In this work, as an alternative, we propose a novel approach for computing the self-diffusion coefficient using deterministic low-rank approximation techniques, as the minimum of a high-dimensional optimization problem. The computed self-diffusion coefficient is then used for the simulation of the cross-diffusion system using an implicit finite volume scheme.

Kalman smoother error bounds in the presence of misspecified measurements

The performance of a discrete-time fixed-interval Kalman smoother in the presence of a misspecified measurement equation is considered. An easily calculable numerical bound for the increment in smoothing error energy is provided in an adversarial non-linear setting by formulating a high-dimensional optimization problem which is solvable via its Lagrange dual, a scalar convex optimization problem. The results are validated by an adversarial learner, implemented as a neural network.

Applying a stochastic quasi-Newton optimizer to least-squares reverse time migration

As we keep moving towards large-scale 3D datasets, inversions based on the wave equation, such as least-squares reverse time migration (LSRTM), must rapidly converge to reduce the computational burden and deliver its well-known advantages, compensating the illumination and providing accurate amplitudes. For that, we explore stochastic optimization methods for LSRTM beyond the minibatch stochastic gradient descent (SGD), considered the state of the art in large-scale machine learning problems. We apply a second-order stochastic method to the LSRTM problem, that uses a quasi-Newton approach, known as the Sum of Functions Optimizer (SFO) algorithm. This method also works well with minibatches of data and has shown good performance on optimization of multi-layer neural networks. It can maintain computational tractability and limit memory requirements even for high dimensional optimization problems. As typical for quasi-Newton methods no adjustment of hyperparameters is required. On our experiments presented here, using synthetic data, we demonstrated that the SFO algorithm shows better results and faster convergence than the minibatch SGD.

Joint sensor registration and multi-object tracking with PHD filter in distributed multi-sensor networks

This paper addresses the joint sensor registration and multi-object tracking in distributed sensor networks wherein the accurate positions of sensors are unknown. Each local sensor runs the probability hypothesis density filter (PHD) in its local coordinate system, having no knowledge about the relative positions of other sensors in the network. The generalized covariance intersection (GCI) fusion rule is employed to fuse the local densities. To tackle the sensor registration problem, a cost function expressing the information discrepancy between local densities is constructed based on the GCI divergence, then the relative positions of sensors are estimated by minimizing such a cost function. As the constructed cost function leads to a high-dimension optimization problem, for computational efficiency, a pairwise sensor registration strategy is proposed which is proved equally sound with the high-dimension registration method but more computationally efficient. Lastly, we demonstrate that the formulated registration method involves a non-convex problem. To solve such a non-convex problem, a two-step optimization method is proposed which can transform the non-convex problem into a convex one. The efficacy of the proposed method is demonstrated in a challenging tracking scenario via numerical experiments.

Substructuring-based dimension-reduction approach for efficient design optimization of high-rise buildings

Economic and safe structural design of high-rise buildings is a high-dimensional optimization problem. Finding the globally optimal solution is challenging, and analysis, design and optimization are very time consuming. This article proposes a new approach, where the search space dimension is reduced through substructuring. The high-dimensional high-rise building optimization problem is converted to a set of low-rise building optimization subproblems of lower dimensions. The subproblems are iteratively solved instead of the original problem. The substructures are analysed and optimally designed, one after another, until convergence, taking into account the effects of other parts of the building to maintain the building’s integrity. Thus, the analysis of a high-rise building is accomplished by analysing a set of low-rise building substructures. Concurrently, the high-dimensional optimization problem is solved through some low-dimensional subproblems. Therefore, building analysis, design and optimization are conducted simultaneously. To demonstrate the efficiency and robustness of the approach, examples from the literature aresolved.

Priority‐aware intelligent device access management for carbon footprint monitoring in sustainable cites and society

Carbon footprint monitoring provides significant basis for computing facilities to improve the computing efficiency and reduce the energy cost, which can enable low/zero carbon computing and facilitate the construction of sustainable cites. Massive sensing devices access to 5G base station to transmit collected carbon emission data, which requires intelligent access management. Fast uplink grant possesses the advantages of reducing signalling overhead and access conflicts while facing the problems of incomplete information and difficulty in guaranteeing priority constraint. This paper examines, the maximum access queuing delay minimization problem is formulated under the long-term service priority constraint and the short-term access management constraint. First, Lyapunov optimization is leveraged to decouple the long-term service priority constraint and short-term access management optimization, and decompose the long-term stochastic optimization problem into a series of short-term deterministic problems. Then, a priority-aware deep Q-network (DQN)-based fast uplink grant access management (PDAC) algorithm is proposed to achieve intelligent access management with differentiated service priority requirements. PDAC utilizes DQN to handle non-convex high-dimensional optimization problem with service priority constraint to achieve intelligent access management and priority awareness. Simulation results demonstrate that PDAC outperforms the existing algorithms in access queuing delay, buffer queue backlog, and priority deficit fluctuation.

Wind Power Prediction Interval Based on Predictive Density Estimation Within a New Hybrid Structure

Wind power prediction interval (WPPI) is the most common technique to represent wind power (WP) uncertainty. This article proposes a novel WPPI approach developed based on predictive density estimation (DE). Unlike most WPPI models in the literature, the proposed model does not need to solve a high-dimensional optimization problem for model training. It optimizes the WPPIs using a single control variable—the bandwidth (BW) of DE—and trains the model directly and noniteratively using the quantiles extracted from the WP predictive density. For predictive DE, a novel application-specific method has been developed based on generalized cross-entropy (GCE). A precise but straightforward technique is designed to determine the optimal BW that results in the optimal WPPIs. The original GCE-based DE problem is also transformed into a convex quadratic programming formulation that can be solved quickly and uniquely. The WPPI model is employed in a new hybrid deterministic/probabilistic WPP (HDPWP) framework. Different from the conventional HDPWP approach that constructs WPPIs based on the point prediction error, the proposed framework incorporates WP point prediction among the predictor variables in the WPPI model, thereby improving performance. The effectiveness of the proposed methods is confirmed through extensive simulations and comparisons using real-world WP generation datasets.

A Novel Self-Adaptive Cooperative Coevolution Algorithm for Solving Continuous Large-Scale Global Optimization Problems

Unconstrained continuous large-scale global optimization (LSGO) is still a challenging task for a wide range of modern metaheuristic approaches. A cooperative coevolution approach is a good tool for increasing the performance of an evolutionary algorithm in solving high-dimensional optimization problems. However, the performance of cooperative coevolution approaches for LSGO depends significantly on the problem decomposition, namely, on the number of subcomponents and on how variables are grouped in these subcomponents. Also, the choice of the population size is still an open question for population-based algorithms. This paper discusses a method for selecting the number of subcomponents and the population size during the optimization process (“on fly”) from a predefined pool of parameters. The selection of the parameters is based on their performance in the previous optimization steps. The main goal of the study is the improvement of coevolutionary decomposition-based algorithms for solving LSGO problems. In this paper, we propose a novel self-adapt evolutionary algorithm for solving continuous LSGO problems. We have tested this algorithm on 15 optimization problems from the IEEE LSGO CEC’2013 benchmark suite. The proposed approach, on average, outperforms cooperative coevolution algorithms with a static number of subcomponents and a static number of individuals.

Two-step estimation of censored quantile regression for duration models with time-varying regressors

Common duration models are characterized by strong homogeneity and thus are highly restrictive in allowing for how regressors affect the conditional duration distribution. In particular, the implied sign and relative marginal quantile effects remain the same over the entire range of the conditional duration distribution, which rules out general heterogeneous effects in duration data. Quantile regression, which offers a flexible and unified framework that allows for general heterogeneous effects, is particularly well suited to duration analysis. Based on the insights behind the accelerated failure time model (AFT) with time-varying regressors (Cox and Oakes, 1984) and the standard quantile regression model (Koenker and Bassett, 1978), Chen (2019) recently developed a quantile regression framework with time-varying regressors. However, Chen’s (2019) estimator is very difficult to compute because the estimation procedure involves a non-convex and nonlinear high dimensional optimization problem due to censoring and the nonlinearity of the quantile function. In this paper I propose an easy-to-implement two-step quantile regression estimator, which significantly reduces the computational burden. The estimator is shown to be consistent and asymptotically normal. Monte Carlo experiments indicate that our estimator perform well in finite samples.

Opposition-based learning equilibrium optimizer with Levy flight and evolutionary population dynamics for high-dimensional global optimization problems

The equilibrium optimizer (EO) is a recently proposed physics-based metaheuristic algorithm inspired by the dynamic mass balance on a control volume. However, EO may encounter local convergence and low accuracy when solving high-dimensional optimization problems. In this work, we propose the opposition-based learning EO algorithm with the Levy flight and the evolutionary population dynamics for high-dimensional global optimization problems, called EOOBLE. Firstly, the opposition-based learning strategy is embedded into the initialization and updating process of EO. Then, the Levy flight strategy is used to improve the exploration ability of EO in the updating mechanism. Moreover, the strategy of evolutionary population dynamics is used in the proposed algorithm to avoid falling into the local optimum. The performance of the proposed algorithm is tested by 25 benchmark functions with dimensions from 100 to 5000, and is compared with 8 state-of-the-art metaheuristic algorithms. The statistical results indicate that the proposed algorithm has better convergence capacity than the compared algorithms. Besides, the proposed algorithm is also compared with different variants of EO, which outperforms the original EO and variants of EO with one or two operators. Therefore, the EOOBLE is a competitive algorithm in solving high-dimensional global optimization problems. Finally, a high-dimensional engineering design problem demonstrates the effectiveness of the EOOBLE.

Optimal Array Geometric Structures for Direct Position Determination Systems

Millimeter-wave (mmWave) fifth-generation (5G) and beyond 5G localization enables the provisioning of extremely accurate positioning information, a feature that has attracted substantial research efforts. In this paper, we contribute to this effort by exploring optimal array geometric structures of direct position determination (DPD) systems, inspired by sensor placement problems that predominantly focus on two-step localization and have not yet been extended to DPD. Specifically, we research an optimal array placement and orientation strategy for two-dimensional (2-D) DPD systems that use sensors equipped with uniform linear arrays (ULA) to localize an agent. The A-optimality criterion in Bayesian optimal (experimental) design theory is invoked to formulate this problem. We use an optimization subproblem that optimizes array orientations when array locations are arbitrary but fixed to tackle this high-dimensional optimization problem. Then the optimization problem is converted into a typical optimal angular separation problem in two-step localization. Experimental results show that judiciously designed array geometric structures can lead to significant performance improvements.