Continuous Piecewise Linear Functions Research Articles

An iterative constrained least squares method for continuous piecewise linear approximation

In this work, a novel iterative least squares method is proposed to approximate nonlinear functions with continuous piecewise linear (CPWL) functions, where the continuity is guaranteed by constrained least squares. A typical least-squares-based method for CPWL fitting consists of two major steps: to perform least squares for a fixed set of breakpoints (i.e. for a specific partition of function domain), and to update breakpoints to reduce the overall fitting error. The proposed method aims to improve the existing CPWL method, the one with canonical representation, by modifying both major steps. Instead of performing ordinary least squares with canonical representation, partitioned least squares and constrained least squares are successively performed to reduce the computational complexity. For the update of breakpoints, an iterative procedure is proposed, where gradient descent with momentum method is employed to improve the convergence characteristics. The advantages of proposed method are illustrated using illustrative examples.

Generating optimal robust continuous piecewise linear regression with outliers through combinatorial Benders decomposition

Using piecewise linear (PWL) functions to model discrete data has applications for example in healthcare, engineering and pattern recognition. Recently, mixed-integer linear programming (MILP) approaches have been used to optimally fit continuous PWL functions. We extend these formulations to allow for outliers. The resulting MILP models rely on binary variables and big- constructs to model logical implications. The combinatorial Benders decomposition (CBD) approach removes the dependency on the big- constraints by separating the MILP model into a master problem of the complicating binary variables and a linear sub problem over the continuous variables, which feeds combinatorial solution information into the master problem. We use the CBD approach to decompose the proposed MILP model and solve for optimal PWL functions. Computational results show that vast speedups can be found using this robust approach, with problem-specific improvements including smart initialization, strong cut generation and special branching approaches leading to even faster solve times, up to more than 12,000 times faster than the standard MILP approach.

An Embedding of ReLU Networks and an Analysis of Their Identifiability

Neural networks with the rectified linear unit (ReLU) nonlinearity are described by a vector of parameters $$\theta $$ and realized as a piecewise linear continuous function $$\varvec{R}_{\theta }: x \in \mathbb {R}^{d} \mapsto \varvec{R}_{\theta }(x) \in \mathbb {R}^{k}$$ . Natural scalings and permutations operations on the parameters $$\theta $$ leave the realization unchanged, leading to equivalence classes of parameters that yield the same realization. These considerations in turn lead to the notion of identifiability—the ability to recover (the equivalence class of) $$\theta $$ from the sole knowledge of its realization $$\varvec{R}_{\theta }$$ . The overall objective of this paper is to introduce an embedding for ReLU neural networks of any depth, $${\varvec{\Phi }}(\theta )$$ , that is invariant to scalings and that provides a locally linear parameterization of the realization of the network. Leveraging these two key properties, we derive some conditions under which a deep ReLU network is indeed locally identifiable from the knowledge of the realization on a finite set of samples $$x_{i} \in \mathbb {R}^{d}$$ . We study the shallow case in more depth, establishing necessary and sufficient conditions for the network to be identifiable from a bounded subset $$\mathcal {X}\subseteq \mathbb {R}^{d}$$ .

Error estimates for the numerical approximation of optimal control problems with nonsmooth pointwise-integral control constraints

Abstract The numerical approximation of an optimal control problem governed by a semilinear parabolic equation and constrained by a bound on the spatial $L^1$-norm of the control at every instant of time is studied. Spatial discretizations of the controls by piecewise constant and continuous piecewise linear functions are investigated. Under finite element approximations, the sparsity properties of the continuous solutions are preserved in a natural way using piecewise constant approximations of the control, but suitable numerical integration of the objective functional and of the constraint must be used to keep the sparsity pattern when using spatially continuous piecewise linear approximations. We also obtain error estimates and finally present some numerical examples.

Neural Network Approaches to Point Lattice Decoding

We characterize the complexity of the lattice decoding problem from a neural network perspective. The notion of Voronoi-reduced basis is introduced to restrict the space of solutions to a binary set. On the one hand, this problem is shown to be equivalent to computing a continuous piecewise linear (CPWL) function restricted to the fundamental parallelotope. On the other hand, it is known that any function computed by a ReLU feed-forward neural network is CPWL. As a result, we count the number of affine pieces in the CPWL decoding function to characterize the complexity of the decoding problem. It is exponential in the space dimension <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , which induces shallow neural networks of exponential size. For structured lattices we show that folding, a technique equivalent to using a deep neural network, enables to reduce this complexity from exponential in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> to polynomial in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> . Regarding unstructured MIMO lattices, in contrary to dense lattices many pieces in the CPWL decoding function can be neglected for quasi-optimal decoding on the Gaussian channel. This makes the decoding problem easier and it explains why shallow neural networks of reasonable size are more efficient with this category of lattices (in low to moderate dimensions).

Fuzzy system F-CalcRank for calculating functions of fuzzy arguments and ranking of fuzzy numbers

Assessing functions of fuzzy arguments and ranking of fuzzy quantities are two key steps in fuzzy modeling and Fuzzy Multicriteria Decision Analysis (FMCDA). Approximate calculations along with the use of centroid index as a defuzzification based ranking methods are a generally accepted approach to applications in the fuzzy environment. This paper presents a novel fuzzy system, F-CalcRank, which is integration of two coupled fuzzy systems: F-Calc (Fuzzy Calculator) and F-Ranking (Fuzzy Ranking). F-Calc allows assessing functions of fuzzy numbers with the use of different approaches: approximate calculations, standard fuzzy arithmetic, and transformation methods. The input values to F-Calc are fuzzy numbers with the following membership functions: triangular and trapezoidal, Gaussian, bell shape, sigmoid, and piece-wise linear continuous or upper semicontinuous membership functions of any complexity, as well as fuzzy linguistic terms of a given term set. F-Ranking system is intended for ranking of a given set of fuzzy numbers, including those, which are inputs and/or outputs of the F-Calc system. F-Ranking includes six ranking methods: three defuzzification based and three pairwise comparison ones. The structure of F-CalcRank as well as input and output information and the user interfaces of both F-Calc and F-Ranking systems, which can also be used independently, are presented. Examples of computing functions of fuzzy arguments and ranking of fuzzy numbers using implemented methods as well as exploring a real case study in agro-ecology with the use of a math model in fuzzy environment are considered. These examples stress the features and novelty of F-CalcRank system as well as presented applied research. The computer modules created within F-CalcRank are a basis for different FMCDA models developed by the authors. F-CalcRank system is intended for university education, research and various applications in engineering and technology.

Piecewise linear iterated function systems on the line of overlapping construction

In this paper we consider iterated function systems (IFS) on the real line consisting of continuous piecewise linear functions. We assume some bounds on the contraction ratios of the functions, but we do not assume any separation condition. Moreover, we do not require that the functions of the IFS are injective, but we assume that their derivatives are separated from zero. We prove that if we fix all the slopes but perturb all other parameters, then for all parameters outside of an exceptional set of less than full packing dimension, the Hausdorff dimension of the attractor is equal to the exponent which comes from the most natural system of covers of the attractor.

Error Estimates of a Continuous Galerkin Time Stepping Method for Subdiffusion Problem

A continuous Galerkin time stepping method is introduced and analyzed for subdiffusion problem in an abstract setting. The approximate solution will be sought as a continuous piecewise linear function in time t and the test space is based on the discontinuous piecewise constant functions. We prove that the proposed time stepping method has the convergence order O(tau ^{1+ alpha }), , alpha in (0, 1) for general sectorial elliptic operators for nonsmooth data by using the Laplace transform method, where tau is the time step size. This convergence order is higher than the convergence orders of the popular convolution quadrature methods (e.g., Lubich’s convolution methods) and L-type methods (e.g., L1 method), which have only O(tau ) convergence for the nonsmooth data. Numerical examples are given to verify the robustness of the time discretization schemes with respect to data regularity.

Frame multiresolution analysis of continuous piecewise linear functions

The Franklin wavelet is constructed using the multiresolution analysis (MRA) generated from a scaling function [Formula: see text] that is continuous on [Formula: see text], linear on [Formula: see text] and [Formula: see text] for every [Formula: see text]. For [Formula: see text] and [Formula: see text], it is shown that if a function [Formula: see text] is continuous on [Formula: see text], linear on [Formula: see text] and [Formula: see text], for [Formula: see text], and generates MRA with dilation factor [Formula: see text], then [Formula: see text]. Conversely, for [Formula: see text], it is shown that there exists a [Formula: see text], as satisfying the above conditions, that generates MRA with dilation factor [Formula: see text]. The frame MRA (FMRA) is useful in signal processing, since the perfect reconstruction filter banks associated with FMRA can be narrow-band. So it is natural to ask, whether the above results can be extended for the case of FMRA. In this paper, for [Formula: see text], we prove that if [Formula: see text] generates FMRA with dilation factor [Formula: see text], then [Formula: see text]. For [Formula: see text], we prove similar results when [Formula: see text]. In addition, for [Formula: see text] we prove that there exists a function [Formula: see text] as satisfying the above conditions, that generates FMRA. Also, we construct tight wavelet frame and wavelet frame for such scaling functions.

Nonconvex multivariate piecewise-linear fitting using the difference-of-convex representation

We address the problem of finding a continuous piecewise-linear (CPWL) approximation of a nonlinear function that satisfies predefined error-tolerances, and keeps the number of polytopes low. We introduce the difference-of-convex (DC) CPWL representation that represents any CPWL function as the difference of two convex CPWL functions. Any convex CPWL function can be represented as the maximum of a set of affine functions, so the polytopes defining a DC CPWL function can be implicitly defined by the affine functions. By simply searching the parameters of affine functions, CPWL approximations can be produced which contain polytopes of any type. We use the DC CPWL representation to develop a CPWL approximation algorithm and prove its finite convergence. We compare the quality of CPWL approximations produced from the proposed algorithm to the best-known existing method. We find that the DC CPWL approximation consistently requires fewer polytopes to meet the same error-tolerance.

Wiener Algebras and Trigonometric Series in a Coordinated Fashion

Let \(W_0(\mathbb {R})\) be the Wiener Banach algebra of functions representable by the Fourier integrals of Lebesgue integrable functions. It is proved in the paper that, in particular, a trigonometric series \(\sum \nolimits _{k=-\infty }^\infty c_k e^{ikt}\) is the Fourier series of an integrable function if and only if there exists a \(\phi \in W_0(\mathbb {R})\) such that \(\phi (k)=c_k\), \(k\in \mathbb {Z}\). If \(f\in W_0(\mathbb {R})\), then the piecewise linear continuous function \(\ell _f\) defined by \(\ell _f(k)=f(k)\), \(k\in \mathbb {Z}\), belongs to \(W_0(\mathbb {R})\) as well. Moreover, \(\Vert \ell _f\Vert _{W_0}\le \Vert f\Vert _{W_0}\). Similar relations are established for more advanced Wiener algebras. These results are supplemented by numerous applications. In particular, new necessary and sufficient conditions are proved for a trigonometric series to be a Fourier series and new properties of \(W_0\) are established.

Robustness analysis and identification for an enzyme-catalytic complex metabolic network in batch culture.

Bioconversion of glycerol to 1,3-propanediol is a promising way to mitigate the shortage of energy. To maximize the production of 1,3-propanediol, it needs to control precisely microbial fermentation process. However, it might consume lots of human and material resources when conducting experimental tests many times. In this study, a nonlinear enzyme-catalytic dynamical system is developed to describe the bioconversion process of glycerol to 1,3-propanediol, especially continuous piecewise linear functions are used as identification parameters. The existence, uniqueness and continuity of solutions are also discussed. Then, considering the fact that the concentration of intracellular substances is difficult to measure in experiments, a new quantitative definition of biological robustness is introduced as a performance index to determine the identification parameters related to intracellular substances. Meanwhile, a two-phase optimization algorithm is constructed to solve the identification model. By comparison with the experimental data, it can be found that the present nonlinear dynamical system can describe the fermentation process very well. Finally, the present nonlinear dynamical system and the corresponding optimal identification parameters might be useful in future studies on the batch culture of glycerol to 1,3-propanediol.

MINLP formulations for continuous piecewise linear function fitting

We consider a nonconvex mixed-integer nonlinear programming (MINLP) model proposed by Goldberg et al. (Comput Optim Appl 58:523–541, 2014. https://doi.org/10.1007/s10589-014-9647-y) for piecewise linear function fitting. We show that this MINLP model is incomplete and can result in a piecewise linear curve that is not the graph of a function, because it misses a set of necessary constraints. We provide two counterexamples to illustrate this effect, and propose three alternative models that correct this behavior. We investigate the theoretical relationship between these models and evaluate their computational performance.

Estimation of Diabetes Prevalence, and Evaluation of Factors Affecting Blood Glucose Levels and Use of Medications in Japan

Background: Diabetes is a noncommunicable disease caused by high levels of blood glucose, and it is currently one of the most important public health problems in the world. It is important to know the prevalence of diabetes, the factors affecting blood glucose levels, and the percentage of people with diabetes taking medications. Data and Methods Data and Methods: We analyzed the distribution of blood glucose levels and prevalence of diabetes using 10,917,173 observations obtained from the JMDC Claims Database in Japan. The factors that may affect blood glucose levels were analyzed by a regression model using 5,472,205 observations. Treatment with diabetes medications was analyzed with 9,932,854 and 5,466,361 observations using a method to approximate the inverse of probability by a continuous piecewise linear function. Results: The prevalence of diabetes in 2019 was estimated to be 9.63% in males and 5.33% in females ages 20 - 79; 10.78% and 7.04% for ages 20 - 89; and 10.93% and 7.65% for ages 20 - 99, respectively. In addition to age and gender, the important variables affecting blood glucose levels were BMI, SBP, Triglyceride, ALT, AST and GGP. The percentage taking medications increased up to a blood glucose level of around 175 mg/dL, but declined over that. Conclusion: The prevalence of diabetes in Japan was estimated using a very large dataset, and considering age, gender, and time trends. Some variables may be effective for controlling blood glucose levels. Nearly half of those at a serious stage of diabetes took no medications. Proper medical care for these individuals is necessary to prevent worsening diabetes and serious complications. Limitations: The dataset was observatory, and did not include those age 80 or over. Revising medical care systems to include those outside of hospitals is necessary; however, practical approaches have not yet been developed.

On the Zeros of Non-Analytic Random Periodic Signals

Abstract In this paper, we investigate the local universality of the number of zeros of a random periodic signal of the form $S_n(t)=\sum _{k=1}^n a_k f(k t)$, where $f$ is a $2\pi -$periodic function satisfying weak regularity conditions and where the coefficients $a_k$ are i.i.d. random variables, which are centered with unit variance. In particular, our results hold for continuous piecewise linear functions. We prove that the number of zeros of $S_n(t)$ in a shrinking interval of size $1/n$ converges in law as $n$ goes to infinity to the number of zeros of a Gaussian process whose explicit covariance only depends on the function $f$ and not on the common law of the random coefficients $(a_k)$. As a byproduct, this entails that the point measure of the zeros of $S_n(t)$ converges in law to an explicit limit on the space of locally finite point measures on $\mathbb R$ endowed with the vague topology. The standard tools involving the regularity or even the analyticity of $f$ to establish such kind of universality results are here replaced by some high-dimensional Berry–Esseen bounds recently obtained in [ 7]. The latter allow us to prove functional Central Limit Theorems in $C^1$ or Lipschitz topology in situations where usual criteria cannot be applied due to the lack of regularity.

A new C0 layerwise wavelet finite element formulation for the static and free vibration analysis of composite plates

In this paper, a new C0 layerwise wavelet finite element is proposed for the static and free vibration analysis of composite plates. The refined zigzag theory is adopted to introduce the zigzag effects in multilayered plate structures by using piecewise linear C0 continuous functions. Then the layerwise wavelet-based BSWI element is derived based on the higher-order plate theory by means of two-dimensional BSWI scaling functions. The proposed model satisfies the conditions of transverse shear stress continuity at the layer interfaces as well as yields to the stress-free boundary conditions on the surface of plate without a shear correction factor. What’s more, the layerwise wavelet-based BSWI element also possesses the advantages of high convergence, high accuracy and reliability with fewer degrees of freedom on account of the excellent approximation property of BSWI. The accuracy and effectiveness of proposed layerwise wavelet-based BSWI element is assessed for static and free vibration analysis of laminated composite and sandwich plates with available 3D elasticity solutions, finite element solutions and other referential solutions in published literatures.

A novel once-data-exchange method for solving transmission and distribution networks coordinated ACOPF model

With the scale increase of distributed generation, the coupling feature between the transmission and distribution networks is further enhanced. The separate optimization mechanisms of the networks cannot adapt to the development of the situation, which would bring some problems to power system operation. Thus, it has become an urgent task for the power system to solve the coordinated Alternating Current Optimal Power Flow (ACOPF) model for the transmission and distribution networks. However, the performance of the traditional centralized method is unsatisfactory in solving such large scale optimization problems for the high computing requirements. In this paper, a novel once-data-exchange method is developed to solve the coordinated ACOPF model. Firstly, the relationship between the optimal objective function and the active power demand at the boundary in the distribution network is described by a piecewise linear continuous function. Secondly, the reactive power demand at the boundary of the distribution network is converted to a piecewise linear continuous function of the active power demand at the boundary. Based on the two functions, the coordinated ACOPF model could be easily simplified to a mixed-integer programming problem. Numerical test results indicate that the proposed method performs better than the traditional methods in accuracy and computational efficiency.

An algorithm for piece-wise indefinite quadratic programming problem

An indefinite quadratic programming problem is a mathematical programming problem which is a product of two linear factors. In this paper, the piece-wise indefinite quadratic programming problem (PIQPP) is considered. Here, the objective function is a product of two continuous piecewise linear functions defined on a non-empty and compact feasible region. In the present paper, the optimality criterion is derived and explained in order to solve PIQPP. While solving PIQPP, we will come across certain variables which will not satisfy the optimality condition. For these variables, cases have been elaborated so as to move from one basic feasible solution to another till we reach the optimality. An algorithmic approach is proposed and discussed for the pr PIQPP problem. A numerical example is presented to decipher the tendered method.

On the Derivation of Continuous Piecewise Linear Approximating Functions

We propose mixed-integer programming models for fitting univariate discrete data points with continuous piecewise linear (PWL) functions. The number of approximating function segments and the locations of break points are optimized simultaneously. The proposed models include linear constraints and convex objective function and, thus, are computationally more efficient than previously proposed mixed-integer nonlinear programming models. We also show how the proposed models can be extended to approximate univariate functions with PWL functions with the minimum number of segments subject to bounds on the pointwise error.

Significance of black hole quasinormal modes: A closer look

It is known that approximating the Regge-Wheeler Potential with step functions significantly modifies the Schwarzschild black hole quasinormal mode spectrum. Surprisingly, this change in the spectrum has little impact on the ringdown waveform. We examine whether this issue is caused by the jump discontinuities and/or the piecewise constant nature of step functions. We show that replacing the step functions with a continuous piecewise linear function does not qualitatively change the results. However, in contrast to previously published results, we discover that the ringdown waveform can be approximated to arbitrary precision using either step functions or a piecewise linear function. Thus, this approximation process provides a new mathematical tool to calculate the ringdown waveform. In addition, similar to normal modes, the quasinormal modes of the approximate potentials seem to form a complete set that describes the entire time evolution of the ringdown waveform. We also examine smoother approximations to the Regge-Wheeler potential, where the quasinormal modes can be computed exactly, to better understand how different portions of the potential impact various regions of the quasinormal mode spectrum.