Nondifferentiable Problems Research Articles

Higher order symmetric duality for multiobjective fractional programming problems over cones

This article studies a pair of higher order nondifferentiable symmetric fractional programming problem over cones. First, higher order cone convex function is introduced. Then using the properties of this function, duality results are set up, which give the legitimacy of the pair of primal dual symmetric model.

Weighted Least Square Design Technique for Hilbert Transformer using Fractional Derivative

In this paper, a new design method for realization of Hilbert transformer (HT) by an all-pass infinite impulse response (IIR) filter is proposed. The design problem is devised as minimization of the weighted least squared errors of the phase response in the frequency domain, where the phase response of denominator polynomial of HT acts as a weighting function. The least squared design techniques usually suffer from large errors at the band edge frequencies. To resolve this issue, our aim is to have the phase response of the all-pass IIR filter closely matched with the desired phase response of HT. The approximation with desired phase response of HT has been achieved by using fractional derivatives constraints (FDCs). In our experiments, it is observed that the design problem using single fractional derivative (FD) has a multimodal behavior in nature. Moreover, highly precise value of order of FD is required, whose exploration task is computationally expensive. Therefore, recently developed heuristic search techniques, also known as swarm-based optimization techniques (SOTs), such as particle swarm optimization (PSO), and its variants, cuckoo search (CS) algorithm, and artificial bee colony (ABC) algorithm, are used for finding the required values. However, these methods are capable of solving non-differentiable and multimodal problems due to their multi-dimensional randomly guided search mechanism. The proposed methodology has gained 59% improvement in phase approximation error as compared to conventional reported techniques.

Bayesian Optimization Based Efficient Layer Sharing for Incremental Learning

Incremental learning is a methodology that continuously uses the sequential input data to extend the existing network’s knowledge. The layer sharing algorithm is one of the representative methods which leverages general knowledge by sharing some initial layers of the existing network. To determine the performance of the incremental network, it is critical to estimate how much the initial convolutional layers in the existing network can be shared as the fixed feature extractors. However, the existing algorithm selects the sharing configuration through improper optimization strategy but a brute force manner such as searching for all possible sharing layers case. This is a non-convex and non-differential problem. Accordingly, this can not be solved using powerful optimization techniques such as the gradient descent algorithm or other convex optimization problem, and it leads to high computational complexity. To solve this problem, we firstly define this as a discrete combinatorial optimization problem, and propose a novel efficient incremental learning algorithm-based Bayesian optimization, which guarantees the global convergence in a non-convex and non-differential optimization. Additionally, our proposed algorithm can adaptively find the optimal number of sharing layers via adjusting the threshold accuracy parameter in the proposed loss function. With the proposed method, the global optimal sharing layer can be found in only six or eight iterations without searching for all possible layer cases. Hence, the proposed method can find the global optimal sharing layers by utilizing Bayesian optimization, which achieves both high combined accuracy and low computational complexity.

A Superfast Super-Resolution Method for Radar Forward-Looking Imaging

The super-resolution method has been widely used for improving azimuth resolution for radar forward-looking imaging. Typically, it can be achieved by solving an undifferentiable regularization problem. The split Bregman algorithm (SBA) is a great tool for solving this undifferentiable problem. However, its real-time imaging ability is limited to matrix inversion and iterations. Although previous studies have used the special structure of the coefficient matrix to reduce the computational complexity of each iteration, the real-time performance is still limited due to the need for hundreds of iterations. In this paper, a superfast SBA (SFSBA) is proposed to overcome this shortcoming. Firstly, the super-resolution problem is transmitted into an regularization problem in the framework of regularization. Then, the proposed SFSBA is used to solve the nondifferentiable regularization problem. Different from the traditional SBA, the proposed SFSBA utilizes the low displacement rank features of Toplitz matrix, along with the Gohberg-Semencul (GS) representation to realize fast inversion of the coefficient matrix, reducing the computational complexity of each iteration from to . It uses a two-order vector extrapolation strategy to reduce the number of iterations. The convergence speed is increased by about 8 times. Finally, the simulation and real data processing results demonstrate that the proposed SFSBA can effectively improve the azimuth resolution of radar forward-looking imaging, and its performance is only slightly lower compared to traditional SBA. The hardware test shows that the computational efficiency of the proposed SFSBA is much higher than that of other traditional super-resolution methods, which would meet the real-time requirements in practice.

E-differentiable minimax programming under E-convexity

In this paper, a new class of minimax programming problems is considered in which the functions involved are E-differentiable. The so-called parametric and nonparametric necessary E-optimality conditions are derived for the considered E-differentiable minimax programming problem. Further, sufficient optimality conditions are established for such nondifferentiable extremum problems under E-convexity hypotheses. Moreover, the example of a nonsmooth minimax programming problem with E-differentiable functions is given to illustrate the aforesaid results. Furthermore, the so-called Mond-Weir E-dual problem and Wolfe E-dual problem are defined for the considered E-differentiable minimax programming problem and several E-duality theorems are established also under appropriate E-convexity hypotheses.

Robust fault estimation for a 3-DOF helicopter considering actuator saturation

Fault estimation plays an important role in real-time monitoring, diagnosis and active fault tolerant control of helicopter system. This paper presents several robust observer based actuator fault estimation designs for an experimental 3-DOF helicopter system with the consideration of potential actuator saturation. In order to deal with the non-differentiable problem that induced by input saturation theoretically, the saturation function is approximated by a continuous function in the fault estimation design process. Both Luenberger observer and unknown input observer are adopted in the actuator fault estimation design scheme. Furthermore, in order to achieve sensor reduction and also avoid inaccurate measurement, simultaneous state and actuator fault estimation design is developed for the helicopter system. An energy-to-energy strategy is applied to ensure the robust performance of the proposed fault estimation approaches, in which approximation error of the saturation function, model uncertainties and external disturbances are all taking into account. In addition, pole placement technique is utilized to adjust the transient response of the estimation errors. Observer gains of the proposed fault estimation designs are derived by solving linear matrix inequalities. Finally, based on Quanser’s 3-DOF helicopter platform, comparative simulation and experimental tests are all carried out to show the effectiveness as well as performance of the proposed fault estimation approaches.

Inexact Cuts in Stochastic Dual Dynamic Programming Applied to Multistage Stochastic Nondifferentiable Problems

In [V. Guigues, SIAM J. Optim., 30 (2020), pp. 407--438], an inexact variant of stochastic dual dynamic programming (SDDP) called ISDDP was introduced which uses approximate (instead of exact with SDDP) primal dual solutions of the problems solved in the forward and backward passes of the method. That variant of SDDP was studied in [V. Guigues, SIAM J. Optim., 30 (2020), pp. 407--438] for linear and for differentiable nonlinear multistage stochastic programs (MSPs). In this paper, we extend ISDDP to nondifferentiable MSPs. We first provide formulas for inexact cuts for value functions of convex nondifferentiable optimization problems. We then combine these cuts with SDDP to describe ISDDP for nondifferentiable MSPs and analyze the convergence of the method. More precisely, for a problem with $T$ stages, we show that for errors bounded from above by $\varepsilon$, the limit superior and limit inferior of sequences of upper and lower bounds on the optimal value of the problem are at most at distance $3 \varepsilon T$ to the optimal value and that for asymptotically vanishing errors ISDDP converges to an optimal policy. Finally, we present the results of encouraging numerical experiments on a multistage nondifferentiable stochastic convex program solved using exact SDDP and the proposed ISDDP.

Optimal correction of the absolute value equations

In this paper, we study the optimum correction of the absolute value equations through making minimal changes in the coefficient matrix and the right hand side vector and using spectral norm. This problem can be formulated as a non-differentiable, non-convex and unconstrained fractional quadratic programming problem. The regularized least squares is applied for stabilizing the solution of the fractional problem. The regularized problem is reduced to a unimodal single variable minimization problem and to solve it a bisection algorithm is proposed. The main difficulty of the algorithm is a complicated constraint optimization problem, for which two novel methods are suggested. We also present optimality conditions and bounds for the norm of the optimal solutions. Numerical experiments are given to demonstrate the effectiveness of suggested methods.

Optimality, duality and saddle point analysis for interval-valued nondifferentiable multiobjective fractional programming problems

This paper is developed to discuss the theoretical aspects of multiobjective fractional problems under interval uncertainty. For this, we have formulated an interval multiobjective fractional model (IVMFP) in which the numerator and denominator of objective functions are assumed to be interval valued functions. Partial ordering is invoked to define LU-Pareto optimal solutions to the problem. Thereafter, necessary optimality conditions of the Fritz John type and Kuhn Tucker type are derived in the classical way. LU-convexity and LU-concavity are used to establish the sufficient optimality criteria and with the same assumptions, the parametric duality theory is discussed. Also, interval valued vector Lagrangian is proposed and the corresponding results are established to locate the saddle point.

Smooth Twin Bounded Support Vector Machine with Pinball Loss

The twin support vector machine improves the classification performance of the support vector machine by solving two smaller quadratic programming problems. However, for the twin support vector machine and some of its variants, the constructed models are usually transformed from the original space into the dual space to obtain the solutions. Meanwhile, the hinge loss function used in above models is sensitive to noise and unstable in resampling. In order to further improve the performance of the twin support vector machine, the pinball loss function is introduced into the twin bounded support vector machine directly, and the non-differentiable problem of the pinball loss function at zero is solved by constructing a smooth approximation function. Based on this, a smooth twin bounded support vector machine model with pinball loss is obtained. The model is solved iteratively in the original space by using the Newton-Armijo method, then a smooth twin bounded support vector machine with pinball loss algorithm is proposed. In the experiments, the proposed twin support vector machine is validated on the UCI datasets, which shows the effectiveness of the proposed algorithm.

Twin-parametric margin support vector machine with truncated pinball loss

In this paper, we propose a novel classifier termed as twin-parametric margin support vector machine with truncated pinball loss (TPin-TSVM), which is motivated by the twin-parametric margin support vector machine (TPMSVM). The proposed TPin-TSVM has the following characteristics. Firstly, it can preserve both sparsity and feature noise insensitivity simultaneously, because it deals with the quantile distance which makes it less sensitive to noises, and most of the correctly classified samples are given equal penalties which makes it have the precious sparsity. Secondly, it is a non-differentiable non-convex optimization problem, we adopt the popular and effective concave–convex procedure (CCCP) to solve it. In each iteration of CCCP, the TPMSVM is utilized as a core of our TPin-TSVM, because it determines two nonparallel hyperplanes by solving two smaller sized quadratic programming problems, which greatly improves the computational speed. Thirdly, we investigate its theoretical properties of noise insensitivity and sparsity, and the proposed TPin-TSVM realizes the between-class distance maximization, within-class scatter and misclassification error minimization together. The experiments on two artificial datasets also verify the properties. We perform numerical experiments on thirty-five benchmark datasets to investigate the validity of our proposed algorithm. Experimental results indicate that our algorithm yields the comparable generalization performance compared with three state-of-the-art algorithms.

Nondifferentiable Multiobjective Programming Problem under Strongly K-Gf-Pseudoinvexity Assumptions

In this paper we consider the introduction of the concept of (strongly) K- G f -pseudoinvex functions which enable to study a pair of nondifferentiable K-G- Mond-Weir type symmetric multiobjective programming model under such assumptions.

Higher-order symmetric duality in nondifferentiable multiobjective fractional programming problem over cone contraints

In this paper, we introduce the definition of higher-order K-(C, α, ρ, d)-convexity/pseudoconvexity over cone and discuss a nontrivial numerical examples for existing such type of functions. The purpose of the paper is to study higher order fractional symmetric duality over arbitrary cones for nondifferentiable Mond-Weir type programs under higher- order K-(C, α, ρ, d)-convexity/pseudoconvexity assumptions. Next, we prove appropriate duality relations under aforesaid assumptions.

Nondifferentiable higher-order duality theorems for new type of dual model under generalized functions

The motivation behind this article is to study a class of nondifferentiable multiobjective fractional programming problem in which each component of objective functions contains a term including the support function of a compact convex set. For a differentiable function, we consider a class of higher order pseudo quasi/ strictly pseudo quasi/weak strictly pseudo quasi- (V, ρ, d)-type-I convex functions. Under these the higher-order pseudo quasi/ strictly pseudo quasi/weak strictly pseudo quasi- (V, ρ, d)-type-I convexity assumptions, we prove the higher-order weak, higher-order strong and higher-order converse duality theorems related to efficient solution.

Duality for a class of second order symmetric nondifferentiable fractional variational problems

The present work frames a pair of symmetric dual problems for second order nondifferentiable fractional variational problems over cone constraints with the help of support functions. Weak, strong and converse duality theorems are derived under second order F-convexity assumptions. By removing time dependency, static case of the problem is obtained. Suitable numerical example is constructed.

Second-order nondifferentiable multiobjective mixed type fractional programming programming problems

The motivation behind this paper is to study a class of nondifferentiable multiobjective fractional programming problem in which each component of objective functions contains a term including the support function of a compact convex set. For a differentiable function, we consider the class of second-order (F,alpha,rho,d)-type-I convex functions. Further, we formulate unified (mixed type) dual models and derive duality relations under aforesaid assumptions.

A Complex Algorithm for Solving a Kind of Stochastic Programming

Considering that the probability distribution of random variables in stochastic programming usually has incomplete information due to a perfect sample data in many real applications, this paper discusses a class of two-stage stochastic programming problems modeling with maximum minimum expectation compensation criterion (MaxEMin) under the probability distribution having linear partial information (LPI). In view of the nondifferentiability of this kind of stochastic programming modeling, an improved complex algorithm is designed and analyzed. This algorithm can effectively solve the nondifferentiable stochastic programming problem under LPI through the variable polyhedron iteration. The calculation and discussion of numerical examples show the effectiveness of the proposed algorithm.

An ecological space based hybrid swarm-evolutionary algorithm for software reliability model parameter estimation

Reliability analysis of the software has attracted a lot of attention of the software developers and researchers due to rapid growing need of software in routine life. The software reliability prediction by mathematical models is entirely centered on the estimation of parameter values, and the parameter estimation of the models poses a non-differential, nonlinear, and multimodal problem. A new algorithm based on the concept of ecological space, method of differential evolution (DE) and intelligent behavior of artificial bee colony (ABC) for optimizing the parameter values has been proposed in this paper. The exploration capability in ABC algorithm has been improved by introducing the concept of ecological space. Ecological space is one of the important factors for evolution and reflects the expansion of individual bee in search space. DE technique provides the diversity of bee’s population and faster convergence. The proposed algorithm has been tested with four standard failure datasets. Proficiency of proposed algorithm is also compared with other meta-heuristic algorithms namely ABC, genetic algorithm and particle swarm optimization. Further validation of proposed algorithm is done through comparing its efficiency with hybrid partilce swarm optimization and gravitational search Algorithm. Simulation results verify that proposed hybrid algorithm is very much effective in field of software reliability estimation and would be a competitive one among meta-heuristic optimization algorithms.

Show, Tell, and Polish: Ruminant Decoding for Image Captioning

The encoder-decoder framework has been the base of popular image captioning models, which typically predicts the target sentence based on the encoded source image one word at a time in sequence. However, such a single-pass decoding framework encounters two problems. First, mistakes in the predicted words cannot be corrected and may propagate to the entire sentence. Second, because the single-pass decoder cannot access the following un-generated words, it can only perform local planning to choose every single word according to the preceding words, while lacks the global planning ability as for maintaining the semantic consistency and fluency of the whole sentence. In order to address the above two problems, in this work, we design a ruminant captioning framework which contains an image encoder, a base decoder, and a ruminant decoder. Specifically, the outputs of the former/base decoder are utilized as the global information to guide the words prediction of the latter/ruminant decoder, in an attempt to mimic human polishing process. We enable jointly training of the whole framework and overcome the non-differential problem of discrete words by designing a novel reinforcement learning based optimization algorithm. Experiments on two datasets (MS COCO and Flickr30 k) demonstrate that our ruminant decoding method can bring significant improvements over traditional single-pass decoding based models and achieves state-of-the-art performance.

Duality for Unified Higher-Order Minimax Fractional Programming with Support Function under Type-I Assumptions

This article is devoted to discussing the nondifferentiable minimax fractional programming problem with type-I functions. We focus our study on a nondifferentiable minimax fractional programming problem and formulate a higher-order dual model. Next, we establish weak, strong, and strict converse duality theorems under generalized higher-order strictly pseudo ( V , α , ρ , d ) -type-I functions. In the final section, we turn our focus to study a nondifferentiable unified minimax fractional programming problem and the results obtained in this paper naturally unify. Further, we extend some previously known results on nondifferentiable minimax fractional programming in the literature.