Projection Neural Network Research Articles

A collaborative neurodynamic approach with two-timescale projection neural networks designed via majorization-minimization for global optimization and distributed global optimization

In this paper, two two-timescale projection neural networks are proposed based on the majorization-minimization principle for nonconvex optimization and distributed nonconvex optimization. They are proved to be globally convergent to Karush–Kuhn–Tucker points. A collaborative neurodynamic approach leverages multiple two-timescale projection neural networks repeatedly re-initialized using a meta-heuristic rule for global optimization and distributed global optimization. Two numerical examples are elaborated to demonstrate the efficacy of the proposed approaches.

An efficient and robust varying-parameter projection neural network for sparse signal reconstruction

Sparse signal reconstruction plays an important role in many practical applications, and therefore, many scholars strive to study its various effective (approximate) solution approaches, including some projection neural network models with finite-time or fixed-time convergence due to their parallel nature and convenience for hardware implementation. However, we experimentally find that noise has a significant influence on the convergence performance of these models. To overcome this problem, in this paper, we propose a new efficient varying-parameter projection neural network model (VPPNN) using the sliding mode control technique and a novel time-varying parameter for sparse signal reconstruction. Theoretical analysis shows that this new model not only has shorter fixed convergence time under certain conditions but also has noise-tolerance when compared to the existing projection neural network models. Numerical simulation experiments (via sparse signal/image reconstruction) are performed to illustrate the superiority of the new proposed model over the existing ones in terms of the convergence rate and robustness.

A Collaborative Neurodynamic Optimization Approach to Distributed Nash-Equilibrium Seeking in Multicluster Games With Nonconvex Functions.

In this article, we propose a collaborative neurodynamic optimization (CNO) method for the distributed seeking of generalized Nash equilibriums (GNEs) in multicluster games with nonconvex functions. Based on an augmented Lagrangian function, we develop a projection neural network for the local search of GNEs, and its convergence to a local GNE is proven. We formulate a global optimization problem to which a global optimal solution is a high-quality local GNE, and we adopt a CNO approach consisting of multiple recurrent neural networks for scattering searches and a metaheuristic rule for reinitializing states. We elaborate on an example of a price-bidding problem in an electricity market to demonstrate the viability of the proposed approach.

Homodyned K-Distribution Parameter Estimation in Quantitative Ultrasound: Autoencoder and Bayesian Neural Network Approaches.

Quantitative ultrasound (QUS) analyzes the ultrasound (US) backscattered data to find the properties of scatterers that correlate with the tissue microstructure. Statistics of the envelope of the backscattered radio frequency (RF) data can be utilized to estimate several QUS parameters. Different distributions have been proposed to model envelope data. The homodyned K-distribution (HK-distribution) is one of the most comprehensive distributions that can model US backscattered envelope data under diverse scattering conditions (varying scatterer number density and coherent scattering). The scatterer clustering parameter ( α ) and the ratio of the coherent to diffuse scattering power ( k ) are the parameters of this distribution that have been used extensively for tissue characterization in diagnostic US. The estimation of these two parameters (which we refer to as HK parameters) is done using optimization algorithms in which statistical features such as the envelope point-wise signal-to-noise ratio (SNR), skewness, kurtosis, and the log-based moments have been utilized as input to such algorithms. The optimization methods minimize the difference between features and their theoretical value from the HK model. We propose that the true value of these statistical features is a hyperplane that covers a small portion of the feature space. In this article, we follow two approaches to reduce the effect of sample features' error. We propose a model projection neural network based on denoising autoencoders to project the noisy features into this space based on this assumption. We also investigate if the noise distribution can be learned by the deep estimators. We compare the proposed methods with conventional methods using simulations, an experimental phantom, and data from an in vivo animal model of hepatic steatosis. The network weight and a demo code are available online at ht.tp://code.sonography.ai.

Impulsive Projection Neural Networks for Variational Inequalities and Sparse Signal Reconstruction Application

Impulsive Projection Neural Networks for Variational Inequalities and Sparse Signal Reconstruction Application

Circuit Implementation of Proximal Projection Neural Networks for Composite Optimization Problems

Existing circuits for composite optimization problems tend to ignore the structure of nonsmooth objectives and lead to less practicability. To this end, with a proximal projection neural network (PPNN) and an inertial proximal projection neural network (IPPNN), this paper presents two novel analog circuit frameworks in presence of a nonsmooth term, where the actions of neuron are simulated via a feedback loop formed by integrator, gradient estimate, proximal operator and other basic operation models. In our circiuts, it shows that the stable output voltages are mapped to optimal solutions. Considering the nonsmooth term as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${{l}_{1}}$</tex-math></inline-formula> -norm, we further design an effective analog circuit of soft thresholding operator. Theoretically, it is proved that both PPNN-based and IPPNN-based analog circuit frameworks have stable output voltages in the sense that the value of electronic components meets a certain mild proportion. Finally, two examples are simulated on Multisim 14.0 platform to validate the effectiveness and practicability of the proposed analog circuits.

Two Novel Noise-Suppression Projection Neural Networks With Fixed-Time Convergence for Variational Inequalities and Applications.

This article proposes two novel projection neural networks (PNNs) with fixed-time ( FIXt ) convergence to deal with variational inequality problems (VIPs). The remarkable features of the proposed PNNs are FIXt convergence and more accurate upper bounds for arbitrary initial conditions. The robustness of the proposed PNNs under bounded noises is further studied. In addition, the proposed PNNs are applied to deal with absolute value equations (AVEs), noncooperative games, and sparse signal reconstruction problems (SSRPs). The upper bounds of the settling time for the proposed PNNs are tighter than the bounds in the existing neural networks. The effectiveness and advantages of the proposed PNNs are confirmed by numerical examples.

Solving Fuzzy Convex Programming Problems via a Projection Neural Network Framework

In this paper, the solution of the fuzzy nonlinear optimization problems (FNLOPs) is given using a projection recurrent neural network (RNN) scheme. Since there is a few research for resolving of FNLOP by RNNs, we describe a new framework to solve the problem. By reformulating the original program to an interval problem and then weighting problem, the Karush–Kuhn–Tucker (KKT) conditions are obtained. Moreover, we utilize the KKT conditions into a RNN as a capable tool to solve the problem. Besides, the global convergence and the Lyapunov stability of the neuro-dynamic model are established. In the final step, some simulation examples are stated to validate the obtained results. Reported results are compared with some other previous neural networks.

Projection neural networks with finite-time and fixed-time convergence for sparse signal reconstruction

Projection neural networks with finite-time and fixed-time convergence for sparse signal reconstruction

An event-triggered collaborative neurodynamic approach to distributed global optimization

In this paper, we propose an event-triggered collaborative neurodynamic approach to distributed global optimization in the presence of nonconvexity. We design a projection neural network group consisting of multiple projection neural networks coupled via a communication network. We prove the convergence of the projection neural network group to Karush–Kuhn–Tucker points of a given global optimization problem. To reduce communication bandwidth consumption, we adopt an event-triggered mechanism to liaise with other neural networks in the group with the Zeno behavior being precluded. We employ multiple projection neural network groups for scattered searches and re-initialize their states using a meta-heuristic rule in the collaborative neurodynamic optimization framework. In addition, we apply the collaborative neurodynamic approach for distributed optimal chiller loading in a heating, ventilation, and air conditioning system.

Two-timescale projection neural networks in collaborative neurodynamic approaches to global optimization and distributed optimization

In this paper, we propose a two-timescale projection neural network (PNN) for solving optimization problems with nonconvex functions. We prove the convergence of the PNN with sufficiently different timescales to a local optimal solution. We develop a collaborative neurodynamic approach with multiple such PNNs to search for global optimal solutions. In addition, we develop a collaborative neurodynamic approach with multiple PNNs connected via a directed graph for distributed global optimization. We elaborate on four numerical examples to illustrate the characteristics of the approaches.

Solving general convex quadratic multi-objective optimization problems via a projection neurodynamic model.

A neural network model is constructed to solve convex quadratic multi-objective programming problem (CQMPP). The CQMPP is first converted into an equivalent single-objective convex quadratic programming problem by the mean of the weighted sum method, where the Pareto optimal solution (POS) are given by diversifying values of weights. Then, for given various values weights, multiple projection neural networks are employded to search for Pareto optimal solutions. Based on employing Lyapunov theory, the proposed neural network approach is established to be stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the single-objective problem. The simulation results also show that the presented model is feasible and efficient.

A Novel Projection Neural Network for Solving a Class of Monotone Variational Inequalities

This article provides a novel projection neural network (PNN) for a category of monotone variational inequality (MVI). For simplifying calculation, the feasible region of MVI is separated into two parts: one is the set on which projection has quantitative expression, and the rest part can be described by inequalities. A novel continuous-time PNN is well designed in this article based on the division and its states are verified to exist and ultimately converge to a solution of MVI. The proposed PNN differs from conventional approaches since it only depends on the projection operator on the sets with projection-quantifiable structure instead of the whole feasible region. The feature determines the PNN here owns the advantage of low amount of calculation and flexibility when encountering diverse MVIs. Compared with existing numerical algorithms, model convergence in this article is established on weaker assumptions and one can monitor its state at every moment. Finally, the effectiveness and the efficiency of PNN are testified by several simulation examples and an application in image fusion.

Modified Gradient Projection Neural Network for Multiset Constrained Optimization

To solve nonlinear optimization problems under multiple set constraints, a modified gradient projection neural network (MGPNN) is proposed and investigated. Different from existing approaches specialized for linear constrained optimizations, such as the gradient-based recurrent neural network or dynamic-parameter zeroing neural network, the MGPNN is intrinsically designed from the perspective of the multiple set constrained optimization (MSCO), which is a more generalized form for the linear constrained optimization. The MGPNN is able to efficiently and conveniently provide a feasible solution to the MSCO problem. Ultimately, compared with existing solution methods, numerical simulations and applications to the control of an underactuated portal crane system are provided for verifications of the robust stability and preponderance of the proposed MGPNN model.

Event-Triggered Cardinality-Constrained Cooling and Electrical Load Dispatch Based on Collaborative Neurodynamic Optimization.

This article addresses event-triggered optimal load dispatching based on collaborative neurodynamic optimization. Two cardinality-constrained global optimization problems are formulated and two event-triggering functions are defined for event-triggered load dispatching in thermal energy and electric power systems. An event-triggered dispatching method is developed in the collaborative neurodynamic optimization framework with multiple projection neural networks and a meta-heuristic updating rule. Experimental results are elaborated to demonstrate the efficacy and superiority of the approach against many existing methods for optimal load dispatching in air conditioning systems and electric power generation systems.

Distributed k-Winners-Take-All Network: An Optimization Perspective.

In this article, we proposed an equivalent formulation of the k-winners-take-all (k-WTA) problem as a constrained optimization problem by including the Laplacian matrix of the undirected connected communication graph to adapt to the distributed computing scenario, where an additional auxiliary variable is introduced. To solve the optimization problem in a distributed fashion, we design projection neural networks by using the convex optimization theory, leading to the emergence of a distributed k-WTA network. Our theoretical analysis shows that the proposed distributed k-WTA network has a globally asymptotically stable equilibrium that is identical to the optimal solution to the optimization problem, that is, the correct k-WTA solution. The effectiveness and advantages, including the extendability to constrained k-WTA problems, of the proposed k-WTA network are demonstrated via simulations.

Chiller Plant Operation Planning via Collaborative Neurodynamic Optimization

A chiller plant is an essential part of a heating, ventilation, and air conditioning system. Chiller plant operation planning is to determine the throughput of active chillers, pumps, and fans in a chiller plant to meet cooling load demands with minimized power consumption. Existing planning methods are limited to chiller plant operation with homogeneous devices subject to constraints for the conservation of energy or with heterogeneous devices without considering the conservation of energy. In this article, a mixed-integer optimization problem is formulated for chiller plant operation planning with heterogeneous devices to minimize power consumption subject to various constraints, including the constraints for the conservation of energy. The formulated problem is reformulated as a global optimization problem and solved via collaborative neurodynamic optimization with multiple projection neural networks. Experimental results based on equipment manufacturers’ specifications are elaborated to demonstrate the significantly higher performance of the proposed approach than four mainstream methods in terms of power consumption wattage.

Jerk-Level Zhang Neurodynamics Equivalency of Bound Constraints, Equation Constraints, and Objective Indices for Cyclic Motion of Robot-Arm Systems.

Equivalency is a powerful approach that can transform an original problem into another problem that is relatively more ready to be resolved. In recent years, Zhang neurodynamics equivalency (ZNE), in the form of neurodynamics or recurrent neural networks (RNNs), has been investigated, abstracted, and proposed as a process that can equivalently solve equations at different levels. After long-term research, we have noticed that the ZNE can not only work with equations, but also inequations. Thus, the ZNE of inequation type is proposed, proved, and applied in this study. The ZNE of inequation type can transform different-level bound constraints into unified-level bound constraints. Applications of the jerk-level ZNE of bound constraints, equation constraints, and objective indices ultimately build up effective time-varying quadratic-programming schemes for cyclic motion planning and control (CMPC) of single and dual robot-arm systems. In addition, as an effective time-varying quadratic-programming solver, a projection neural network (PNN) is introduced. Experimental results with single and dual robot-arm systems substantiate the correctness and efficacy of ZNE and especially the ZNE of inequation type. Comparisons with conventional methods also exhibit the superiorities of ZNE.

A Collaborative Neurodynamic Approach to Distributed Global Optimization

In this article, we present a collaborative neurodynamic approach to distributed optimization with nonconvex functions. We develop a recurrent neural network (RNN) group by connecting individual projection neural networks through a communication network. We prove the convergence of the RNN group to the local optimal solutions of a given distributed optimization problem. We propose a collaborative neurodynamic optimization system with multiple RNN groups for scattered searches and a metaheuristic rule for reinitializing the neuronal states upon their local convergence. We elaborate on three numerical examples to demonstrate the efficacy of the proposed approach to distributed global optimization in the presence of nonconvexity.

Utilizing a projection neural network to convex quadratic multi‐objective programming problems

SummaryIn this paper, we propose a projection neural network model for solving convex quadratic multi‐objective optimization problem (CQMOP). The CQMOP is first converted into an equivalent convex nonlinear programming problem by the means of the weighted sum method, where the Pareto optimal solutions are calculated via different values of weights. A neural network model is then constructed for solving the obtained convex problem. It is shown that the presented neural network is stable in the sense of Lyapunov and is globally convergent. Simulation results are given to illustrate the global convergence and performance of the suggested model. Both theoretical and numerical approaches are studied. Numerical results are in good agreement with the proved theoretical concepts.