Nondifferentiable Functions Research Articles

Continuous Nowhere Differentiable Function with Fractal Properties Defined in Terms of Q2-Representation

We construct a continuous nowhere monotone and nondifferentiable function depending on a single parameter q0 ϵ (0; 1). For functions from this continual class, we describe their structural, variational, fractal, and integrodifferential properties.

Development of a genetic algorithm and NONMEM workbench for automating and improving population pharmacokinetic/pharmacodynamic model selection.

The current approach to selection of a population PK/PD model is inherently flawed as it fails to account for interactions between structural, covariate, and statistical parameters. Further, the current approach requires significant manual and redundant model modifications that heavily lend themselves to automation. Within the discipline of numerical optimization it falls into the "local search" category. Genetic algorithms are a class of algorithms inspired by the mathematics of evolution. GAs are general, powerful, robust algorithms and can be used to find global optimal solutions for difficult problems even in the presence of non-differentiable functions, as is the case in the discrete nature of including/excluding model components in search of the best performing mixed-effects PK/PD model. A genetic algorithm implemented in an R-based NONMEM workbench for identification of near optimal models is presented. In addition to the GA capabilities, the workbench supports modeling efforts by: (1) Organizing and displaying models in tabular format, allowing the user to sort, filter, edit, create, and delete models seamlessly, (2) displaying run results, parameter estimates and precisions, (3) integrating xpose4 and PsN to facilitate generation of model diagnostic plots and run PsN scripts, (4) running regression models between post-hoc parameter estimates and covariates. This approach will further facilitate the scientist to shift efforts to focus on model evaluation, hypotheses generation, and interpretation and applications of resulting models.

Intermittency of Riemann’s non-differentiable function through the fourth-order flatness

Riemann’s non-differentiable function is one of the most famous examples of continuous but nowhere differentiable functions, but it has also been shown to be relevant from a physical point of view. Indeed, it satisfies the Frisch–Parisi multifractal formalism, which establishes a relationship with turbulence and implies some intermittent nature. It also plays a surprising role as a physical trajectory in the evolution of regular polygonal vortices that follow the binormal flow. With this motivation, we focus on one more classic tool to measure intermittency, namely, the fourth-order flatness, and we refine the results that can be deduced from the multifractal analysis to show that it diverges logarithmically. We approach the problem in two ways: with structure functions in the physical space and with high-pass filters in the Fourier space.

Time-Dependent Reliability Analysis Based on Point-Evolution Kernel Density Estimation: Comprehensive Approach with Continuous and Shock Deterioration and Maintenance

Civil infrastructure may degrade due to the adverse effects of continuous damage (e.g., reinforcement corrosion) and sudden shocks (e.g., earthquakes) during its service life. Many studies have been conducted in the field of reliability-informed life-cycle assessment, but there is still a need for a general and efficient method to assess the time-dependent performance of aging structures by considering different deterioration scenarios and maintenance actions in a unified manner. Some of the traditional methods may have difficulties in handling multiple deteriorations, nonlinear models, a large number of uncertainties, scenarios of nondifferentiable performance functions, and combined effects of deterioration and maintenance. This paper develops a novel approach for a time-dependent reliability analysis based on the proposed point-evolution kernel density estimation (PKDE) method and equivalent extreme performance function. The proposed approach allows consideration of various uncertainties (e.g., external loads, deterioration scenarios, and maintenance models) and the associated correlation effects. In the proposed approach, both the progressive deterioration and sudden damages are considered in the modeling of the performance function. Besides, different types of maintenance schemes are assessed. The equivalent performance function is established, and the proposed PKDE method is used to address the first passage problem and nondifferentiable performance function within a time-dependent reliability analysis. An illustrative example is made to demonstrate the feasibility and accuracy of the proposed PKDE method. The computational results using the proposed method are verified by comparison with those from Monte Carlo simulations (MCS).

Computationally Efficient Nonlinear Model Predictive Control Using the L1 Cost-Function.

Model Predictive Control (MPC) algorithms typically use the classical L cost function, which minimises squared differences of predicted control errors. Such an approach has good numerical properties, but the L norm that measures absolute values of the control errors gives better control quality. If a nonlinear model is used for prediction, the L norm leads to a difficult, nonlinear, possibly non-differentiable cost function. A computationally efficient alternative is discussed in this work. The solution used consists of two concepts: (a) a neural approximator is used in place of the non-differentiable absolute value function; (b) an advanced trajectory linearisation is performed on-line. As a result, an easy-to-solve quadratic optimisation task is obtained in place of the nonlinear one. Advantages of the presented solution are discussed for a simulated neutralisation benchmark. It is shown that the obtained trajectories are very similar, practically the same, as those possible in the reference scheme with nonlinear optimisation. Furthermore, the L norm even gives better performance than the classical L one in terms of the classical control performance indicator that measures squared control errors.

Smoothing quantile regression for a distributed system

Quantile regression has become a popular alternative to least squares regression for providing a comprehensive description of the response distribution, and robustness against heavy-tailed error distributions. However, the nonsmooth quantile loss poses new challenges to distributed estimation in both computation and theoretical development. To address this challenge, we use a convolution-type smoothing approach and its Taylor expression to transform the nondifferentiable quantile loss function into a convex quadratic loss function, which admits a fast and scalable algorithm to perform optimization under massive and high-dimensional data. The proposed distributed estimators are both computationally and communication efficient. Moreover, only the gradient information is communicated at each iteration. Theoretically, we show that, after a certain number of iterations, the resulting estimator is statistically as efficient as the global estimator without any restriction on the number of machines. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

Distributed generalized Nash equilibrium seeking algorithm for nonsmooth aggregative games

In this paper, nonsmooth aggregative games are investigated, where the nondifferentiable cost function of every player depends on its own decision as well as the aggregate of the decisions of all players. Moreover, in the problem, the local constraints of players are general and heterogeneous convex sets, and the decisions of players are coupled by linear constraints. For purpose of distributed seeking of the variational generalized Nash equilibrium (GNE) of the game, a distributed algorithm is developed for players. In the algorithm, the dynamic average consensus is used for the estimation of the aggregate of decisions to obtain the approximation of subgradients of cost functions. Besides, the convergence of the algorithm to the variational GNE is analyzed. Finally, simulation examples illustrate the algorithm.

The envelope theorem, Euler and Bellman equations, without differentiability

We extend the standard Bellman's theory of dynamic programming and the theory of recursive contracts with forward-looking constraints of Marcet and Marimon (2019) to encompass non-differentiability of the value function associated with non-unique solutions or multipliers. The envelope theorem provides the link between the Bellman equation and the Euler equations, but it may fail to do so if the value function is non-differentiable. We introduce an envelope selection condition which restores this link. In standard single-agent dynamic programming, ignoring the envelope selection condition may result in inconsistent multipliers, but not in non-optimal outcomes. In recursive contracts it can result in inconsistent promises and non-optimal outcomes. Planner problems with recursive preferences are a special case of recursive contracts and, therefore, solutions can be dynamically inconsistent if they are not unique. A recursive method of solving dynamic optimization problems with non-differentiable value function involves expanding the co-state and imposing the envelope selection condition.

Learning per-machine linear dispatching rule for heterogeneous multi-machines control

ABSTRACT This paper proposes a per-machine linear dispatching rule learning approach to improve the scheduling of re-entrant flow shop such as semiconductor fab. Finding an optimal schedule of a complex manufacturing system is intractable; hence, a dispatching rule as a heuristic approach is widely used in actual practice. Also, to develop a good dispatching rule, an automated methodology for developing heuristics, also known as a hyper-heuristic, has been studied extensively. However, most of the literature has focused on finding a single-sophisticated dispatching rule, in which every machine uses the same rule. Such an approach often shows suboptimal performance when the optimal dispatching rule is different on each machine. To solve this problem, we introduce a simple and effective per-machine dispatching rule learning approach, in which each machine has one linear dispatching rule that is optimised by the Gradient-based Evolutionary Strategy (GES). This method is sample-efficient and can be applied to non-differentiable objective functions such as average Cycle Time. The proposed approach was mainly compared to two popular methods based on Genetic Programming (GP) and the Genetic Algorithm (GA) on a four-station and eight-machine re-entrant flow shop. Numerical results show that the proposed approach outperforms widely used methods.

On generalization of E-convex and composite functions

In this paper the concepts of some generalizations of E-convexity such as logarithmic, and exponential E-convexity are introduced for real-valued functions defined on a Banach space and their relationships with known concepts have been discussed. Further, some applications to optimization of non-differentiable objective functions have been proposed. Also, it is proved that the composition of an affine and a G-convex functional is G-convex, where G stands for convex, quasi-convex, r-quasi convex and m-quasi convex.

On the Hausdorff dimension of Riemann’s non-differentiable function

Recent findings show that the classical Riemann's non-differentiable function has a physical and geometric nature as the irregular trajectory of a polygonal vortex filament driven by the binormal flow. In this article, we give an upper estimate of its Hausdorff dimension. We also adapt this result to the multifractal setting. To prove these results, we recalculate the asymptotic behavior of Riemann's function around rationals from a novel perspective, underlining its connections with the Talbot effect and Gauss sums, with the hope that it is useful to give a lower bound of its dimension and to answer further geometric questions.

Subgradient averaging for multi-agent optimisation with different constraint sets

We consider a multi-agent setting with agents exchanging information over a possibly time-varying network, aiming at minimising a separable objective function subject to constraints. To achieve this objective we propose a novel subgradient averaging algorithm that allows for non-differentiable objective functions and different constraint sets per agent. Allowing different constraints per agent simultaneously with a time-varying communication network constitutes a distinctive feature of our approach, extending existing results on distributed subgradient methods. To highlight the necessity of dealing with a different constraint set within a distributed optimisation context, we analyse a problem instance where an existing algorithm does not exhibit a convergent behaviour if adapted to account for different constraint sets. For our proposed iterative scheme we show asymptotic convergence of the iterates to a minimum of the underlying optimisation problem for step sizes of the form ηk+1, η>0. We also analyse this scheme under a step size choice of ηk+1, η>0, and establish a convergence rate of O(lnkk) in objective value. To demonstrate the efficacy of the proposed method, we investigate a robust regression problem and an ℓ2 regression problem with regularisation.

Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information

Deep spiking neural network (DSNN) is a promising computational model towards artificial intelligence. It benefits from both the DNNs and SNNs through a hierarchy structure to extract multiple levels of abstraction and the event-driven computational manner to provide ultra-low-power neuromorphic implementation, respectively. However, how to efficiently train the DSNNs remains an open question because of the non-differentiable spike function that prevents the traditional back-propagation (BP) learning algorithm directly applied to DSNNs. Here, inspired by the findings from the biological neural networks, we address the above-mentioned problem by introducing neural oscillation and spike-phase information to DSNNs. Specifically, we propose an Oscillation Postsynaptic Potential (Os-PSP) and phase-locking active function, and further put forward a new spiking neuron model, namely Resonate Spiking Neuron (RSN). Based on the RSN, we propose a Spike-Level-Dependent Back-Propagation (SLDBP) learning algorithm for DSNNs. Experimental results show that the proposed learning algorithm resolves the problems caused by the incompatibility between the BP learning algorithm and SNNs, and achieves state-of-the-art performance in single spike-based learning algorithms. This work investigates the contribution of introducing biologically inspired mechanisms, such as neural oscillation and spike-phase information to DSNNs and providing a new perspective to design future DSNNs.

Training Spiking Neural Networks with Accumulated Spiking Flow

The fast development of neuromorphic hardwares promotes Spiking Neural Networks (SNNs) to a thrilling research avenue. Current SNNs, though much efficient, are less effective compared with leading Artificial Neural Networks (ANNs) especially in supervised learning tasks. Recent efforts further demonstrate the potential of SNNs in supervised learning by introducing approximated backpropagation (BP) methods. To deal with the non-differentiable spike function in SNNs, these BP methods utilize information from the spatio-temporal domain to adjust the model parameters. With the increasing of time window and network size, the computational complexity of spatio-temporal backpropagation augments dramatically. In this paper, we propose a new backpropagation method for SNNs based on the accumulated spiking flow (ASF), i.e. ASF-BP. In the proposed ASF-BP method, updating parameters does not rely on the spike train of spiking neurons but leverage accumulated inputs and outputs of spiking neurons over the time window, which reduces the BP complexity significantly. We further present an adaptive linear estimation model to approach the dynamic characteristics of spiking neurons statistically. Experimental results demonstrate that with our proposed ASF-BP method, light-weight convolutional SNNs achieve superior performances compared with other spike-based BP methods on both non-neuromorphic (MNIST, CIFAR10) and neuromorphic (CIFAR10-DVS) datasets. The code is available at https://github.com/neural-lab/ASF-BP.

Adaptive fixed-time tracking control for a class of nonlinear pure-feedback systems: A relative threshold event-triggered strategy

In this paper, a fixed-time controller under the mechanism of event-trigger is designed for a class of nonlinear pure-feedback systems with non-differentiable non-affine functions. By properly modeling non-affine terms, the limitation of the partial derivatives of non-affine functions is eliminated. In our design process, we first develop a fixed-time adaptive controller using decoupling method. Then, a relative threshold event-trigger mechanism (ETM) is introduced in Section 3.1. The proposed controller can not only stabilize the system within a fixed-time, but also save communication resources more effectively. Lastly, the feasibility of the proposed control scheme is verified by two simulation examples.

Inverse problems for nonlinear quasi-variational hemivariational inequalities with application to obstacle problems of elliptic type

In this paper we study an inverse problem governed by a constrained nonlinear elliptic quasi-variational hemivariational inequality. The inequality involves two nondifferentiable functions which directly depend on solutions. We solve the direct problem and obtain the properties of weak closedness and uniform boundedness of solutions, which develops some existing results in literature. Then, using a regularized optimization method, we deal with an inverse problem to seek the identification of parameter functions appearing in the leading operator and constraint set. Finally, we apply the abstract result to a concrete inverse problem of identifying the parameter and obstacle functions in elliptic obstacle problems with the p-Laplace operator and mixed boundary conditions.

Neural-network-based adaptive tracking control for nonlinear pure-feedback systems subject to periodic disturbance

ABSTRACT This paper presents an adaptive neural control to solve the tracking problem of a class of pure-feedback systems with non-differentiable non-affine functions in the presence of unknown periodically time-varying disturbances. To handle with the design difficulty from non-affine structure of pure-feedback system, a continuous and positive control gain function is constructed to model the periodically disturbed non-affine function as a form that facilitates the control design. As a result, the non-affine function is not necessary to be differentiable with respect to control variables or input. In addition, the bounds of non-affine function are unknown functions, and some appropriate compact sets are introduced to investigate the bounds of non-affine function so as to cope with the difficulty from these unknown bounds. It is proven that the closed-loop control system is semi-globally uniformly ultimately bounded by choosing the appropriate design parameters. Finally, comparative simulations are provided to illustrate the effectiveness of the proposed control scheme.

Hadamard Semidifferential, Oriented Distance Function, and some Applications

<p style='text-indent:20px;'>The <i>Hadamard semidifferential calculus</i> preserves all the operations of the classical differential calculus including the chain rule for a large family of non-differentiable functions including the continuous convex functions. It naturally extends from the <inline-formula><tex-math id="M1">\begin{document}$ n $\end{document}</tex-math></inline-formula>-dimensional Euclidean space <inline-formula><tex-math id="M2">\begin{document}$ \operatorname{\mathbb R}^n $\end{document}</tex-math></inline-formula> to subsets of topological vector spaces. This includes most function spaces used in <i>Optimization</i> and the <i>Calculus of Variations</i>, the metric groups used in <i>Shape and Topological Optimization</i>, and functions defined on submanifolds.</p><p style='text-indent:20px;'>Certain set-parametrized functions such as the <i>characteristic function</i> <inline-formula><tex-math id="M3">\begin{document}$ \chi_A $\end{document}</tex-math></inline-formula>of a set <inline-formula><tex-math id="M4">\begin{document}$ A $\end{document}</tex-math></inline-formula>, the <i>distance function</i> <inline-formula><tex-math id="M5">\begin{document}$ d_A $\end{document}</tex-math></inline-formula> to <inline-formula><tex-math id="M6">\begin{document}$ A $\end{document}</tex-math></inline-formula>, and the <i>oriented (signed) distance function</i> <inline-formula><tex-math id="M7">\begin{document}$ b_A = d_A-d_{ \operatorname{\mathbb R}^n\backslash A} $\end{document}</tex-math></inline-formula> can be used to identify a space of subsets of <inline-formula><tex-math id="M8">\begin{document}$ \operatorname{\mathbb R}^n $\end{document}</tex-math></inline-formula> with a metric space of set-parametrized functions. Many geometrical properties of domains (convexity, outward unit normal, curvatures, tangent space, smoothness of boundaries) can be expressed in terms of the analytical properties of <inline-formula><tex-math id="M9">\begin{document}$ b_A $\end{document}</tex-math></inline-formula> and a simple intrinsic differential calculus is available for functions defined on hypersurfaces without appealing to local bases or Christoffel symbols.</p><p style='text-indent:20px;'>The object of this paper is to extend the use of the Hadamard semidifferential and of the oriented distance function from finite to infinite dimensional spaces with some selected illustrative applications from shapes and geometries, plasma physics, and optimization.</p>

Spike neuron optimization using deep reinforcement learning

Deep reinforcement learning (DRL) which involved reinforcement learning and artificial neural network allows agents to take the best possible actions to achieve goals. Spiking Neural Network (SNN) faced difficulty in training due to the non-differentiable spike function of spike neuron. In order to overcome the difficulty, Deep Q network (DQN) and Deep Q learning with normalized advantage function (NAF) are proposed to interact with a custom environment. DQN is applied for discrete action space whereas NAF is implemented for continuous action space. The model is trained and tested to validate its performance in order to balance the firing rate of excitatory and inhibitory population of spike neuron by using both algorithms. Training results showed both agents able to explore in the custom environment with OpenAI Gym framework. The trained model for both algorithms capable to balance the firing rate of excitatory and inhibitory of the spike neuron. NAF achieved 0.80% of the average percentage error of rate of difference between target and actual neuron rate whereas DQN obtained 0.96%. NAF attained the goal faster than DQN with only 3 steps taken for actual output neuron rate to meet with or close to target neuron firing rate.

Quasi-Duality Result and Linearization in Multiobjective Quasiconvex Programming

In this paper, multiobjective optimization problems with nondifferentiable quasiconvex functions are considered. We obtain some duality results and a linear representation for the considered problems. Since the well-known strong duality result is not valid for the problems, we present a weaker form of it, named quasi-strong duality result.