Nondifferentiable Problems Research Articles

Approximate efficient solutions for nondifferentiable multiobjective optimization problems with an infinite number of constraints

Approximate efficient solutions for nondifferentiable multiobjective optimization problems with an infinite number of constraints

Differential evolution-based topology optimization using Karhunen-Loève expansion

Topology optimization is a widely adopted structural optimization approach that aims to maximize desired performance based on mathematical and physical principles. Gradient-based methods are common for obtaining optimal configurations; however, they struggle with problems involving numerous local optima and non-differentiable problems, which prevent the derivation of design sensitivity. To address these issues, this study explores the application of the Success History-Based Differential Evolution with Linear Population Size Reduction and Semi-Parameter Adaptation, which is regarded as LSHADE-SPA algorithm, known for its strong search capabilities. Nevertheless, its computational cost makes it impractical for topology optimization, especially as the number of design variables increases. To overcome this challenge, a novel structure representation method utilizing the Karhunen-Loève (KL) expansion is constructed. Applying truncated KL expansion instead of conventional density methods to represent structural configurations reduces the number of design variables. This article introduces a differential evolution-based topology optimization method utilizing truncated KL expansion and validates its effectiveness through applications in various structural and fluid problems. The method is particularly suited to highly nonlinear problems, such as negative Poisson's ratio structures and micro-mixer designs. The findings here indicate that differential evolution with truncated KL expansion can serve as an efficient and robust topology optimization method, proving particularly effective for solving optimization problems that are challenging for conventional gradient-based approaches. Notably, the proposed methodology not only delivers superior results but also alleviates computational burdens, thereby advancing the prospects of topology optimization in various engineering applications.

Optimality results for nondifferentiable multiobjective fractional programming problems under E-B-invexity

In this article, we study nonconvex multiobjective fractional programming problems involving E-differentiable functions (MFPE). We establish the E-Karush–Kuhn–Tucker (E-KKT) sufficient E-optimality conditions for nonsmooth vector optimization problems under the assumption of E-B-invexity. To demonstrate the validity of the derived results, we provide an example where the involved functions exhibit E-B-invexity.

Optimality and duality in nondifferentiable minimax programming problems involving $$\epsilon $$-quasi solutions

Optimality and duality in nondifferentiable minimax programming problems involving $$\epsilon $$-quasi solutions

Knowledge Graph Guided Neural Machine Translation with Dynamic Reinforce-selected Triples

Previous methods incorporating knowledge graphs (KGs) into neural machine translation (NMT) adopt a static knowledge utilization strategy, that introduces many useless knowledge triples and makes the useful triples difficult be utilized by NMT. To address this problem, we propose a KG guided NMT model with dynamic reinforce-selected triples. The proposed methods could dynamically select the different useful knowledge triples for different source sentences. Specifically, the proposed model contains two components: 1) knowledge selector, that dynamically selects useful knowledge triples for a source sentence, and 2) knowledge guided NMT (KgNMT), that utilizes the selected triples to guide the translation of NMT. Meanwhile, to overcome the non-differentiable problem and guide the training procedure, we propose a policy gradient strategy to encourage the model to select useful triples and improve the generation probability of gold target sentence. Various experimental results show that the proposed method can significantly outperform the baseline models in both translation quality and handling the entities.

Improvement of Convergence of One- and Two-Step MSM Iteration Methods for Nondifferentiable Nonlinear Complementarity Problems

Improvement of Convergence of One- and Two-Step MSM Iteration Methods for Nondifferentiable Nonlinear Complementarity Problems

Optimality and Unified Duality for E-differentiable Vector Optimization Problems over Cones involving Generalized E-convexity

This paper explores a new approach to solve a nondifferentiable vector optimization problem over cones by means of an operator E : Rn → Rn which renders differentiability to the considered problem. Some new generalized convexity notions are introduced and employed to obtain necessary and sufficient KKT-type optimality conditions. Further, a unified dual is associated with the considered problem encapsulating both Mond-Weir and Wolfe type duals in the same apparatus and duality results are proved.

ZeroGrads: Learning Local Surrogates for Non-Differentiable Graphics

Gradient-based optimization is now ubiquitous across graphics, but unfortunately can not be applied to problems with undefined or zero gradients. To circumvent this issue, the loss function can be manually replaced by a "surrogate" that has similar minima but is differentiable. Our proposed framework, ZeroGrads , automates this process by learning a neural approximation of the objective function, which in turn can be used to differentiate through arbitrary black-box graphics pipelines. We train the surrogate on an actively smoothed version of the objective and encourage locality, focusing the surrogate's capacity on what matters at the current training episode. The fitting is performed online, alongside the parameter optimization, and self-supervised, without pre-computed data or pre-trained models. As sampling the objective is expensive (it requires a full rendering or simulator run), we devise an efficient sampling scheme that allows for tractable run-times and competitive performance at little overhead. We demonstrate optimizing diverse non-convex, non-differentiable black-box problems in graphics, such as visibility in rendering, discrete parameter spaces in procedural modelling or optimal control in physics-driven animation. In contrast to other derivative-free algorithms, our approach scales well to higher dimensions, which we demonstrate on problems with up to 35k interlinked variables.

Optimal control of an elastic–rigid obstacle problem

In this paper, we consider the optimal control problem of an elastic–rigid obstacle problem. The force is considered as the control, and the corresponding solution to the elastic–rigid obstacle problem is taken to be the state. An approximate optimization problem is proposed by regularizing the original non-differentiable constrained problem, and the connection between the two formulations is established through some convergence results. Existence of optimal pairs is proved and necessary optimality conditions are derived.

PREFACE — SPECIAL ISSUE ON FRACTALS AND LOCAL FRACTIONAL CALCULUS: RECENT ADVANCES AND FUTURE CHALLENGES

Fractal geometry plays an important role in the description of the characteristics of nature. Local fractional calculus, a new branch of mathematics, is used to handle the non-differentiable problems in mathematical physics and engineering sciences. The local fractional inequalities, local fractional ODEs and local fractional PDEs via local fractional calculus are studied. Fractional calculus is also considered to express the fractal behaviors of the functions, which have fractal dimensions. The interesting problems from fractional calculus and fractals are reported. With the scaling law, the scaling-law vector calculus via scaling-law calculus is suggested in detail. Some special functions related to the classical, fractional, and power-law calculus are also presented to express the Kohlrausch–Williams–Watts function, Mittag-Leffler function and Weierstrass–Mandelbrot function. They have a relation to the ODEs, PDEs, fractional ODEs and fractional PDEs in real-world problems. Theory of the scaling-law series via Kohlrausch–Williams–Watts function is suggested to handle real-world problems. The hypothesis for the tempered Xi function is proposed as the Fractals Challenge, which is a new challenge in the field of mathematics. The typical applications of fractal geometry are proposed in real-world problems.

High Efficient 3D Convolution Feature Compression

In this paper, a high efficient 3D convolution feature compression method is proposed. This method is mainly used to compress the three-dimensional convolution deep features extracted by video analysis. Gather the compressed features to the cloud server instead of all video features. This method can solve the problem that the data aggregation of video big data analysis requires a large amount of network bandwidth. Quantization is a common method for feature compression, but the existing quantization-based methods often carry out model training and quantization in stages, which makes the robustness of quantization results poor. To solve this problem, the method proposed in this paper is to apply the feature quantization operation directly to the network and train it with the analysis task, and use the parameter iterative optimization method to solve the non-differentiable problem of quantization operation. Different from the deep features extracted from images or single objects, the 3D convolution features extracted from video clips have high time-domain redundancy. In this paper, by serializing the three-dimensional convolution features, the time-domain prediction coding method is used to remove the time-domain redundancy of the three-dimensional convolution features, to improve the feature compression ratio. The experimental results show that this method can only use 1 bit to represent the elements in the three-dimensional convolution deep feature. When the analysis accuracy loss is no more than 1%, the feature compression ratio can reach 4500 times compared with the original feature data, and the data transmission can be reduced by 96%.

An Enhanced Genetic Algorithm using Directional-Based Crossover and normal mutation For Global Optimization Problems

Global optimization has been employed in many practical modeling processes. Using gradient methods to solve optimization problems may be computationally inefficient and time-consuming, particularly when convexity or differentiability is not guaranteed. On the other hand, nature-inspired techniques offer an effective gradient-free approach for solving complex, non-convex, or non-differentiable problems. Genetic algorithms are one of the most effective and widely used nature-inspired techniques. However, canonical genetic algorithms do not always guarantee convergence to the optimum point owing to the stochastic nature of the genetic operators, and typically require more work to ensure convergence and increase performance. Improving the genetic operators remains an open issue and usually involves a trade-off between the speed of convergence and searchability. In this study, we propose an enhanced genetic algorithm that relies on directional-based crossover and normal mutation operators to increase the speed of convergence while preserving searchability. The proposed algorithm is evaluated using a set of 40 typical benchmark functions in two dimensions. In addition, to examine its performance at higher dimensions, 16 functions from the test set were tested at 10 and 100 dimensions. The evaluation results of the proposed algorithm are compared to the outcomes of three modern optimization algorithms, namely (Whale optimization algorithm, Teacher-Learner based algorithm, and Covariance matrix adaptation evolution strategy). The results revealed that the proposed algorithm outperformed the conventional algorithms at lower dimensions in all test functions and showed a relatively better performance than the other algorithms at higher dimensions.

An Application of Infinite Programming on Desirability Functions

When assessing the quality of a system or product, it is necessary to take all responses into account and optimize them in a concurrent manner to find the factor levels that satisfy the overall system, process, or product properties to solve the robust design problem. This problem can be solved as a multi-response optimization problem. There are many methods suggested to solve this problem based on different disciplines like multi-objective optimization. In this study, we improve the theory of nondifferentiable desirability functions' optimization for which the so-called gradient-based methods are not useful. In this study, we propose an infinite programming approach for nondifferentiable desirability functions including more than one nondifferentiable point. We employed DNLP model of GAMS/BARON which is a nondifferentiable solver, however, the solution of more than one nondifferentiable point problem is resulted as infeasible. We also tested MATLAB/NOMAD which is a derivative-free solver for MINLP problems, however, MATLAB/NOMAD also did not succeed and could not solve this nondifferentiable problem. Lastly, we use a genetic algorithm that is implemented under MATLAB and it also cannot find a feasible solution. We use an example that is solved by different desirability functions approaches before and show that the desirability functions approach with more than one nondifferentiable point is a good alternative to the ones in the literature. We present the conclusion and future studies at the end of the paper.

Exploration of differentiability in a proton computed tomography simulation framework

Objective. Gradient-based optimization using algorithmic derivatives can be a useful technique to improve engineering designs with respect to a computer-implemented objective function. Likewise, uncertainty quantification through computer simulations can be carried out by means of derivatives of the computer simulation. However, the effectiveness of these techniques depends on how ‘well-linearizable’ the software is. In this study, we assess how promising derivative information of a typical proton computed tomography (pCT) scan computer simulation is for the aforementioned applications. Approach. This study is mainly based on numerical experiments, in which we repeatedly evaluate three representative computational steps with perturbed input values. We support our observations with a review of the algorithmic steps and arithmetic operations performed by the software, using debugging techniques. Main results. The model-based iterative reconstruction (MBIR) subprocedure (at the end of the software pipeline) and the Monte Carlo (MC) simulation (at the beginning) were piecewise differentiable. However, the observed high density and magnitude of jumps was likely to preclude most meaningful uses of the derivatives. Jumps in the MBIR function arose from the discrete computation of the set of voxels intersected by a proton path, and could be reduced in magnitude by a ‘fuzzy voxels’ approach. The investigated jumps in the MC function arose from local changes in the control flow that affected the amount of consumed random numbers. The tracking algorithm solves an inherently non-differentiable problem. Significance. Besides the technical challenges of merely applying AD to existing software projects, the MC and MBIR codes must be adapted to compute smoother functions. For the MBIR code, we presented one possible approach for this while for the MC code, this will be subject to further research. For the tracking subprocedure, further research on surrogate models is necessary.

Magnetic Inversion through a Modified Adaptive Differential Evolution

In recent decades, differential evolution (DE) has been employed to address a diverse range of nonlinear, nondifferentiable, and nonconvex optimization problems. In this study, we introduce an enhanced adaptive differential evolution algorithm to address the inversion problem associated with magnetic data. The primary objective of the inversion process is to minimize the discrepancy between observed data and predicted data derived from the inverted model. So, the contributions of this paper include the following two points. First, a new mechanism for generating crossover rate (CR) is proposed, which tends to reduce the CR values corresponding to vectors with better objective function values. Second, a new mutation strategy with direction information is proposed to expedite convergence. Additionally, modifications were made to the adjustment of the regularization factor to prevent it from becoming too minimal, thereby preserving its efficacy. The proposed algorithm is validated through synthetic models and a field example. Results from synthetic models demonstrate that our method is superior to and competitive with the original adaptive DE in both solution quality and convergence velocity. For the field example, the Inverted models align closely with the drill-well information.

Lightweight spiking neural network training based on spike timing dependent backpropagation

Spiking neural networks are energy efficient and biological interpretability, communicating through sparse, asynchronous spikes, which makes them suitable for neuromorphic hardware. However, due to the nature of binary weights and spike trains in time-coded binarized spiking neural networks, their forward propagation may cause neurons to not fire spikes, and their backward propagation has non-differentiable problems. Moreover, the current use of deep and complex network structures generates a large number of redundant parameters. Therefore, we need effective methods to improve energy efficiency without reducing accuracy. We propose a dynamic threshold model that can reduce the number of dead neurons. We combine the backpropagation algorithm and the spike timing dependent plasticity algorithm to avoid the non-differentiable problem. We propose a neuron pruning strategy based on adaptive firing time threshold. This pruning strategy prunes 267 neurons in a network of 600 neurons, reducing the network size and obtaining a more compact network structure. The energy efficiency is improved by 0.55×, while the classification accuracy is lost by 1.1%.

A Partial Approximate Least Absolute Deviation-Based Identification Algorithm for a Multivariable Closed-Loop System with Spike Noise

Considering the stability and safety of industrial production, the open-loop system cannot meet the requirements of industrial production, so more and more attention has been paid to multivariable closed-loop system identification. This study focuses on the identification of a multivariate closed-loop system with spike noise in which the model order of the feedback channel is lower than that of the forward channel. Combining principal component analysis (PCA), which is used to eliminate the correlation between the data matrix elements, and the derivable approximate least absolute criterion function, partial approximate least absolute deviation (PALAD) for multivariable closed-loop system identification is proposed. By introducing a deterministic function to replace the absolute value in partial approximate least absolute deviation, the non-differentiable problem of the least absolute deviation function can be solved, and the identifiability of the multivariable closed-loop system in the case mentioned above is theoretically verified. Simulation experiments show the validity of the PALAD algorithm. Compared with the partial least squares (PLS) method, PALAD can effectively restrain the spike noise that follows an SαS distribution and shows stronger robustness when white and spike noises exist simultaneously.

A double decomposition based coevolutionary algorithm for distributed multi-objective OPF solution

Due to the vast and complex nature of the power systems, the study of distributed solution to multi-objective OPF (MO-OPF) problem is vital. However, the existing distributed MO-OPF approaches are based on mathematical programming techniques. The tedious repetition of algorithmic executions is inevitable. Moreover, they can hardly solve the non-differential problem. In this paper, the coevolutionary multi-objective evolutionary algorithm (MOEA), combining the idea of decomposition, is introduced to solve the distributed MO-OPF problem. The decomposition first occurs on decision variables. After segmenting, multiple subpopulations can coevolve in a distributed manner under the support of a new proposed distributed fitness evaluation method. Further, an objective decomposition (OBD) method is applied to the proposed distributed MO-OPF approach. The problem is decomposed again based on objective functions. By this OBD method, the fitness assignment problem is effectively alleviated and a more extensively covered pareto front can be obtained. The experimental results on two systems of different scales first show the effectiveness of the proposed distributed approach and then demonstrate the excellence of the OBD method in large-scale system.

A Comparative Study of Metaheuristic Optimization Algorithms for Solving Engineering Design Problems

Metaheuristic optimization algorithms (Nature-Inspired Optimization Algorithms) are a class of algorithms that mimic the behavior of natural systems such as evolution process, swarm intelligence, human activity and physical phenomena to find the optimal solution. Since the introduction of meta-heuristic optimization algorithms, they have shown their profound impact in solving the high-scale and non-differentiable engineering problems. This paper presents a comparative study of the most widely used nature-inspired optimization algorithms for solving engineering classical design problems, which are considered challenging. The teen metaheuristic algorithms employed in this study are, namely, Artificial Bee Colony (ABC), Ant Colony Optimization (ACO), Biogeography Based Optimization Algorithm (BBO), Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES), Cuckoo Search algorithm (CS), Differential Evolution (DE), Genetic Algorithm (GA), Grey Wolf Optimizer (GWO), Gravitational Search Algorithm (GSA) and Particle Swarm Optimization (PSO). The efficiency of these algorithms is evaluated on teen popular engineering classical design problems using the solution quality and convergence analysis, which verify the applicability of these algorithms to engineering classical constrained design problems. The experimental results demonstrated that all the algorithms provide a competitive solution.

ALADA: A lite automatic data augmentation framework for industrial defect detection

Industrial defect detection is a critical and challenging task in the quality control of manufacturing production. Competent in feature extraction and pattern recognition, deep learning shows great power for classifying and locating defects in industrial products. However, insufficient data records and diverse categories of defects restrict the detection accuracy of data-driven neural networks. One direction to solve such a problem is data augmentation, which aims at generating synthetic copies from existing data and improving the generalizability of detection models. However, confirming a suitable augmentation policy involves either human experience or substantial experiments for augmentation parameters. To address these challenges, in this work, a lite automatic data augmentation (ALADA) framework is proposed to jointly optimize the data augmentation policies and the neural network for industrial defect detection. First, a lite search space is formulated to efficiently sample augmentation policies and generate augmented images for joint optimization. To reduce the hyperparameter tuning efforts for retraining with searched policies, a three-step bi-level optimization scheme is proposed to replace the retraining strategy and update the model and augmentation parameter alternately. To solve the non-differentiable problem in the joint optimization scheme, policy gradient sampling is implemented to estimate the gradient flow efficiently. Experimental results on three industrial defect detection datasets, namely, Tianchi-TILE, GC10-DET, and NEU-DET, reveal that our proposed automatic augmentation framework outperforms the state-of-the-art augmentation methods and effectively improves the accuracy of the baseline defect detection model. The proposed ALADA scheme also alleviates the missed detection of defects in four practical industrial circumstances: textured background, uneven brightness, low contrast, and intraclass difference.