Nondifferentiable Problems Research Articles

Efficient decoupling-assisted evolutionary/metaheuristic framework for expensive reliability-based design optimization problems

Reliability-based design optimization (RBDO) algorithm is to minimize the objective under the probabilistic factors. While gradient-based and classical evolutionary RBDO algorithms provide promising performance on simple optimization problems, they are likely to perform poorly on challenging problems, including the multimodal functions, discrete design spaces, non-differential problems, etc. This paper proposes a unified framework to improve the performance of existing RBDO algorithms for complex RBDO problems. Our framework is based on three new strategies: generalized decoupling evolutionary and metaheuristic RBDO framework, particle’s memory saving strategy, and adaptive fractional-order equilibrium optimizer algorithm. The proposed algorithm is characterized by a decoupling strategy to enable the parallel operation of the inner reliability computation and outer deterministic optimization, a particle’s memory saving strategy to provide effective guidance from the previous iteration, and the adaptive fractional-order equilibrium optimizer algorithm to enhance the search efficiency and global convergence capacity. To evaluate the performance of the proposed algorithm, a wide range of experiments are conducted on different types of use cases. The experimental results demonstrate that our algorithm provides superior performance over other comparative algorithms.

Numerical Solution of the Absolute Value Equations Using Two Matrix Splitting Fixed Point Iteration Methods

The absolute value equations (AVEs) are significant nonlinear and non-differentiable problems that arise in the optimization community. In this article, we provide two new iteration methods for determining AVEs. These two approaches are based on the fixed point principle and splitting of the coefficient matrix with three extra parameters. The convergence of these procedures is also presented using some theorems. The validity of our methodologies is demonstrated via numerical examples.

Image Denoising Using a Compressive Sensing Approach Based on Regularization Constraints.

In remote sensing applications and medical imaging, one of the key points is the acquisition, real-time preprocessing and storage of information. Due to the large amount of information present in the form of images or videos, compression of these data is necessary. Compressed sensing is an efficient technique to meet this challenge. It consists in acquiring a signal, assuming that it can have a sparse representation, by using a minimum number of nonadaptive linear measurements. After this compressed sensing process, a reconstruction of the original signal must be performed at the receiver. Reconstruction techniques are often unable to preserve the texture of the image and tend to smooth out its details. To overcome this problem, we propose, in this work, a compressed sensing reconstruction method that combines the total variation regularization and the non-local self-similarity constraint. The optimization of this method is performed by using an augmented Lagrangian that avoids the difficult problem of nonlinearity and nondifferentiability of the regularization terms. The proposed algorithm, called denoising-compressed sensing by regularization (DCSR) terms, will not only perform image reconstruction but also denoising. To evaluate the performance of the proposed algorithm, we compare its performance with state-of-the-art methods, such as Nesterov’s algorithm, group-based sparse representation and wavelet-based methods, in terms of denoising and preservation of edges, texture and image details, as well as from the point of view of computational complexity. Our approach permits a gain up to 25% in terms of denoising efficiency and visual quality using two metrics: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

Sufficient optimality condition and duality of nondifferentiable minimax ratio constraint problems under (p, r)-ρ-(η, θ)-invexity

Abstract There are several classes of decision-making problems that explicitly or implicitly prompt fractional programming problems. Portfolio selection problems, agricultural planning, information transfer, numerical analysis of stochastic processes, and resource allocation problems are just a few examples. The huge number of applications of minimax fractional programming problems inspired us to work on this topic. This paper is concerned with a nondifferentiable minimax fractional programming problem. We study a parametric dual model, corresponding to the primal problem, and derive the sufficient optimality condition for an optimal solution to the considered problem. Further, we obtain the various duality results under (p, r)-ρ-(η, θ)-invexity assumptions. Also, we identify a function lying exclusively in the class of (−1, 1)-ρ-(η, θ)-invex functions but not in the class of (1, −1)-invex functions and convex function already existing in the literature. We have given a non-trivial model of nondifferentiable minimax problem and obtained its optimal solution using optimality results derived in this paper.

Nondifferentiable generalized minimax fractional programming under (Ф,ρ)-invexity

In this paper, a class of nonconvex nondifferentiable generalized minimax fractional programming problems is considered. Sufficient optimality conditions for the considered nondifferentiable generalized minimax fractional programming problem are established under the concept of (?, ?)-invexity. Further, two types of dual models are formulated and various duality theorems relating to the primal minimax fractional programming problem and dual problems are established. The results established in the paper generalize and extend several known results in the literature to a wider class of nondifferentiable minimax fractional programming problems. To the best of our knowledge, these results have not been established till now.

Optimality and duality results for (h,φ)-nondifferentiable multiobjective programming problems with (h,φ) – (b,f,ρ) -convex functions

Generalized algebraic operations introduced by Ben-Tal [5] are used to define new classes of generalized convex functions, namely (h,?)?(b,F,?) -convex functions and generalized (h,?)?(b,F,?)-convex functions in the vectorial case. Further, optimality and duality results are proved for the considered (h,?)- nondifferentiable multiobjective programming problem under assumptions that the functions involved are (generalized) (h,?)- (b,F,?)-convex.

Duality theorems for nondifferentiable semi-infinite interval-valued optimization problems with vanishing constraints

In this paper, we study the duality theorems of a nondifferentiable semi-infinite interval-valued optimization problem with vanishing constraints (IOPVC). By constructing the Wolfe and Mond–Weir type dual models, we give the weak duality, strong duality, converse duality, restricted converse duality, and strict converse duality theorems between IOPVC and its corresponding dual models under the assumptions of generalized convexity.

IWA: Integrated gradient‐based white‐box attacks for fooling deep neural networks

The widespread application of deep neural network (DNN) techniques is being challenged by adversarial examples—the legitimate input added with imperceptible and well-designed perturbation that can fool DNNs easily in the DNN testing/deploying stage. Previous white-box adversarial example generation algorithms used the Jacobian gradient information to add the perturbation. This imprecise and inexplicit information can cause unnecessary perturbation when generating adversarial examples. This paper aims to address this issue. We first propose to apply the more informative and distilled gradient information, namely, integrated gradient, to generate adversarial examples. To further make the perturbation more imperceptible, we propose to employ the restriction combination of L 0 and L 1 / L 2 second, which can restrict the total perturbation and the perturbation points simultaneously. Meanwhile, to address the nondifferentiable problem of L 1 , we explore a proximal operation of L 1 third. On the basis of these three works, we propose two Integrated gradient-based White-box Adversarial example generation algorithms (IWA): Integrated gradient-based Finite Point Attack (IFPA) and Integrated gradient-based Universe Attack (IUA). IFPA is suitable for situations where there are a determined number of points to be perturbed. IUA is suitable for situations where no perturbation point number is preset to obtain more adversarial examples. We verify the effectiveness of the proposed algorithms on both structured and unstructured data sets, and compare them with five baseline generation algorithms. The results show that our proposed algorithms craft adversarial examples with more imperceptible perturbation and satisfactory crafting rate. L 2 restriction is suitable for unstructured data sets and L 1 restriction performs better in the structured data set.

Capacity Characterization for Reconfigurable Intelligent Surfaces Assisted Multiple-Antenna Multicast

The reconfigurable intelligent surface (RIS), which consists of a large number of passive and low-cost reflecting elements, has been recognized as a revolutionary technology to enhance the performance of future wireless networks. This paper considers an RIS assisted multicast transmission, where a base station (BS) with multiple-antenna multicasts common message to multiple single-antenna mobile users (MUs) under the assistance of an RIS. An equivalent channel model for the considered multicast transmission is analyzed, and then an optimization problem for the corresponding channel capacity is formulated to obtain the optimal covariance matrix and phase shifts. In order to solve the above non-convex and non-differentiable problem, this paper first exploits the gradient descent method and alternating optimization, to approach the locally optimal solution for any number of MUs. Then, this paper considers a special case, which can obtain the global optimal solution, and shows the sufficient and necessary condition for this special case. Finally, the order growth of the maximal capacity is obtained when the numbers of the reflecting elements, the BS antennas, and the MUs go to infinity.

A posteriori error estimates for semilinear optimal control problems

In two and three dimensional Lipschitz, but not necessarily convex, polytopal domains, we devise and analyze a reliable and efficient a posteriori error estimator for a semilinear optimal control problem; control constraints are also considered. We consider a fully discrete scheme that discretizes the state and adjoint equations with piecewise linear functions and the control variable with piecewise constant functions. The devised error estimator can be decomposed as the sum of three contributions which are associated to the discretization of the state and adjoint equations and the control variable. We extend our results to a scheme that approximates the control variable with piecewise linear functions and also to a scheme that approximates the solution to a nondifferentiable optimal control problem. We illustrate the theory with two and three-dimensional numerical examples.

Proximal Gradient-Type Algorithms for a Class of Bilevel Programming Problems

A class of proximal gradient-type algorithm for bilevel nonlinear nondifferentiable programming problems with smooth substructure is developed in this paper. The original problem is approximately reformulated by explicit slow control technique to a parameterized family function which makes full use of the information of smoothness. At each iteration, we only need to calculate one proximal point analytically or with low computational cost. We prove that the accumulation iterations generated by the algorithms are solutions of the original problem. Moreover, some results of complexity of the algorithms are presented in convergence analysis. Numerical experiments are implemented to verify the efficiency of the proximal gradient algorithms for solving this kind of bilevel programming problems.

An outer–inner linearization method for non-convex and nondifferentiable composite regularization problems

An outer–inner linearization method for non-convex and nondifferentiable composite regularization problems

Improving the variability of urban traffic microsimulation through the calibration of generative parameter models

The calibration of urban traffic microsimulation models is addressed in this paper, focusing on the multivariate distribution of traffic features. This allows to take into account day-to-day variability and possibly complex statistical dependencies, enabling robust validation and the computation of more accurate statistics. Variability in traffic models can be obtained, in principle, by suitably tuning origin-destination demand flows, but this becomes problematic for microsimulation models when complex multivariate distributions are considered, due to excessive dimensionality and nonlinearity. Thus, here we investigate how complex multivariate behavior can be achieved by suitably varying only a subset of the parameters ruling the microsimulation dynamics, without imposing any distribution on the demand flows. To this purpose, rather than looking for fixed values of the parameters, we exploit distribution models having the structure of mixtures of uniforms, through which the simulator parameter values can be randomly sampled, thus replacing the simulator inner randomization mechanism. We formalize the optimization of the mixture parameters through a maximum mean discrepancy principle involving the simulator dynamics, which leads to a large-dimensional non-differentiable problem that we solve through a method based on cross-entropy. Preliminary experiments concerning the application of the proposed methodology to the SUMO simulator show how it is possible to capture quite complex multivariate distributions of target flows varying only four parameters.

A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models

Sequential recommender systems (SRS) have become a research hotspot in recent studies. Because of the requirement in capturing user's dynamic interests, sequential neural network based recommender models often need to be stacked with more hidden layers (e.g., up to 100 layers) compared with standard collaborative filtering methods. However, the high network latency has become the main obstacle when deploying very deep recommender models into a production environment. In this paper, we argue that the typical prediction framework that treats all users equally during the inference phase is inefficient in running time, as well as sub-optimal in accuracy. To resolve such an issue, we present SkipRec, an adaptive inference framework by learning to skip inactive hidden layers on a per-user basis. Specifically, we devise a policy network to automatically determine which layers should be retained and which layers are allowed to be skipped, so as to achieve user-specific decisions. To derive the optimal skipping policy, we propose using gumbel softmax and reinforcement learning to solve the non-differentiable problem during backpropagation. We perform extensive experiments on three real-world recommendation datasets, and demonstrate that SkipRec attains comparable or better accuracy with much less inference time.

Joint application of multi-object beetle antennae search algorithm and BAS-BP fuel cost forecast network on optimal active power dispatch problems

The multi-object optimal active power dispatch (MOAPD) problem can realize the optimized operation of electric systems by reducing the fuel cost, power loss and exhaust emission. In this paper, the Pareto front (PF) and the best-compromise scheme (BCS) of MOAPD problems are determined by novel multi-object beetle antennae search algorithm (NMBAS). The proposed NMBAS algorithm, which innovatively integrates the advanced ideas of widely-used NSGA-Ⅱ and multi-goal PSO (MPSO) algorithms, achieves better BCS-quality, PF-uniformity and PF-diversity. More notably, further research indicates that several winning elite solutions (WES) with smaller goal values are distributed around the current BCS of NMBAS. Therefore, a brand new BAS-BP fuel cost forecast network with less prediction-error and time-cost is put forward to seek the potential WES schemes. With BP network as main body and modified by beetle antennae search (BAS) algorithm, the BAS-BP fuel cost forecast model achieves more than six higher-quality WES schemes which are superior to the traditional BCS solution. The superiorities of proposed NMBAS algorithm and BAS-BP fuel cost forecast network are intuitively demonstrated by dual-objective and triple-objective MOAPD experiments. In general, the joint application of NMBAS algorithm and BAS-BP forecast network successfully solves the non-differentiable power grid optimization problems.

Numerical solution of some systems of nonlinear algebraic equations

The problem of numerical solution of systems of nonlinear algebraic equations with even powers is studied in this paper. Nondifferentiable (nonsmooth) unconstrained minimization problems are associated with the system under consideration, with objective functions, based on discrete l 1 - and l ∞ - norm, respectively, by using these two norms as proximity criteria, and the considered system is solved by minimizing the sum of absolute values of residuals, in the case of l 1 - norm, or by minimizing the maximal residual, in the case of l ∞ - norm, respectively. Also, a “differentiable” unconstrained minimization problem (least squares problem) is associated with the considered system of nonlinear equations, with objective function, based on the discrete l 2 -norm, and the considered system is solved by minimizing the sum of squares of the residuals. Subgradients of the objective functions of unconstrained minimization problems are calculated, and a subgradient method for solving the associated problems is presented. The iterative gradient method for solving the corresponding “differentiable” discrete least squares problem, based on l 2 - norm, is also outlined, and gradients of the objective function of the corresponding “differentiable” unconstrained minimization problems are calculated.

Zero duality gap conditions via abstract convexity

Using tools provided by the theory of abstract convexity, we extend conditions for zero duality gap to the context of non-convex and nonsmooth optimization. Mimicking the classical setting, an abstract convex function is the upper envelope of a family of abstract affine functions (being conventional vertical translations of the abstract linear functions). We establish new conditions for zero duality gap under no topological assumptions on the space of abstract linear functions. In particular, we prove that the zero duality gap property can be fully characterized in terms of an inclusion involving (abstract) -subdifferentials. This result is new even for the classical convex setting. Endowing the space of abstract linear functions with the topology of pointwise convergence, we extend several fundamental facts of functional/convex analysis. This includes (i) the classical Banach–Alaoglu–Bourbaki theorem (ii) the subdifferential sum rule, and (iii) a constraint qualification for zero duality gap which extends a fact established by Borwein, Burachik and Yao (2014) for the conventional convex case. As an application, we show with a specific example how our results can be exploited to show zero duality for a family of non-convex, non-differentiable problems.

Solving vector interval-valued optimization problems with infinite interval constraints via integral-type penalty function

This paper aims at establishing necessary optimality conditions for a nondifferentiable multiobjective interval-valued optimization problem with infinite interval-valued constraints, and then deriving the equivalence of the weakly LU efficient solution between the primal optimization problem and its integral-type penalized optimization problem, which provides us supporting theories when we solve this kind of optimization problem by means of penalty function method. The equivalence of the weakly LU efficiency is established under some suitable conditions. Some examples are provided to illustrate the advantages of our results in some cases.

Some results related to constrained non-differentiable (non-smooth) pseudolinear minimization problems

Some results related to constrained non-differentiable (non-smooth) pseudolinear minimization problems

A Zeno-Free Self-Triggered Approach to Practical Fixed-Time Consensus Tracking With Input Delay

This article considers the practical fixed-time self-triggered consensus tracking problem of delayed multiagent networks (MANs) subject to external disturbances under undirected topology and directed topology. The fixed-time consensus implies that the consensus is reached in a finite time and the convergence time is independent of initial conditions under the nonlinear consensus protocols. A self-triggered control (STC) strategy is developed based on the event-triggered control (ETC) strategy. For the ETC strategy, the nonlinear controllers and the measurement errors are designed based on the hyperbolic tangent function to avoid a nondifferential problem and Zeno behavior. To avoid continuous monitoring, the STC strategy is presented. Furthermore, the minimal interevent interval is strictly positive, which implies that no Zeno behavior occurs in the STC strategy. Finally, a numerical example is presented to verify the availability of the algorithms.