Linear Function Approximation Research Articles

Risk Constrained Navigation Using MILP-MPC Formulation

This paper presents a novel approach to deal with chance constrained navigation, i.e., a problem where a robot has to plan its path among obstacles in the presence of noise and perturbations, subject to a maximum probability of failure. An improved formulation allows the problem to be framed as a single stage optimization using the MILP-MPC framework through the use of a piecewise linear approximation of the non-linear function that represent the chance constraints. Time is included in the optimization to enable more efficient risk allocation. A comparison with state-of-the-art algorithm shows advantages of the proposed approach.

Simulation-based decision support framework for dynamic ambulance redeployment in Singapore

ObjectiveDynamic ambulance redeployment policies tend to introduce much more flexibilities in improving ambulance resource allocation by capitalizing on the definite geospatial-temporal variations in ambulance demand patterns over the time-of-the-day and day-of-the-week effects. A novel modelling framework based on the Approximate Dynamic Programming (ADP) approach leveraging on a Discrete Events Simulation (DES) model for dynamic ambulance redeployment in Singapore is proposed in this paper. MethodsThe study was based on the Singapore’s national Emergency Medical Services (EMS) system. Based on a dataset comprising 216,973 valid incidents over a continuous two-years study period from 1 January 2011–31 December 2012, a DES model for the EMS system was developed. An ADP model based on linear value function approximations was then evaluated using the DES model via the temporal difference (TD) learning family of algorithms. The objective of the ADP model is to derive approximate optimal dynamic redeployment policies based on the primary outcome of ambulance coverage. ResultsConsidering an 8min response time threshold, an estimated 5% reduction in the proportion of calls that cannot be reached within the threshold (equivalent to approximately 8000 dispatches) was observed from the computational experiments. The study also revealed that the redeployment policies which are restricted within the same operational division could potentially result in a more promising response time performance. Furthermore, the best policy involved the combination of redeploying ambulances whenever they are released from service and that of relocating ambulances that are idle in bases. ConclusionThis study demonstrated the successful application of an approximate modelling framework based on ADP that leverages upon a detailed DES model of the Singapore’s EMS system to generate approximate optimal dynamic redeployment plans. Various policies and scenarios relevant to the Singapore EMS system were evaluated.

Recursive least-squares temporal difference learning for adaptive traffic signal control at intersection

This paper presents a new method to solve the scheduling problem of adaptive traffic signal control at intersection. The method involves recursive least-squares temporal difference (RLS-TD(λ)) learning that is integrated into approximate dynamic programming. The learning mechanism of RLS-TD(λ) is to make an adaptation of linear function approximation by updating its parameters based on environmental feedback. This study investigates the method implementation after modeling a traffic dynamic system at intersection in discrete time. In the model, different traffic control schemes regarding signal phase sequence are considered, especially the defined adaptive phase sequence (APS). By simulating traffic scenarios, RLS-TD(λ) is superior to TD(λ) for updating functional parameters in the approximation, and APS outperforms other conventional control schemes on reducing traffic delay. By comparing with other traffic signal control algorithms, the proposed algorithm yields satisfying results in terms of traffic delay and computation time.

The Delivery Dispatching Problem with Time Windows for Urban Consolidation Centers

This paper addresses the dispatching problem faced by an urban consolidation center. The center receives orders according to a stochastic arrival process and dispatches them in batches for the last-mile distribution. The operator of the center aims to find the cost-minimizing consolidation policy, depending on the orders at hand, preannounced orders, and stochastic arrivals. We present this problem as a variant of the delivery dispatching problem that includes dispatch windows and define a corresponding Markov decision model. Larger instances of the problem suffer from intractably large state-, outcome-, and action spaces. We propose an approximate dynamic programming (ADP) algorithm that can handle such instances, using a linear value function approximation to estimate the downstream costs. To design the value function approximation, we construct various sets of basis functions, numerically evaluate their suitability, and discuss the properties of good basis functions for the dispatching problem. Numerical experiments on toy-sized instances show that the best set of basis functions approximates the optimal values with an error of less than 3%. To cope with large action spaces, we formulate an integer linear program to be used within our ADP algorithm. We evaluate the performance of ADP policies against four benchmark policies: two heuristic policies, a direct cost minimization policy, and a post-decision rollout policy. We test the performance of ADP on a variety of networks. ADP consistently outperforms the benchmark policies, performing particularly well when there is sufficient flexibility in dispatch times. The online appendix is available at https://doi.org/10.1287/trsc.2017.0773 .

Optimal Data Scheduling and Admission Control for Backscatter Sensor Networks

This paper studies the data scheduling and admission control problem for a backscatter sensor network (BSN). In the network, instead of initiating their own transmissions, the sensors can send their data to the gateway just by switching their antenna impedance and reflecting the received RF signals. As such, we can reduce remarkably the complexity, the power consumption, and the implementation cost of sensor nodes. Different sensors may have different functions, and data collected from each sensor may also have a different status, e.g., urgent or normal, and thus we need to take these factors into account. Therefore, in this paper, we first introduce a system model together with a mechanism in order to address the data collection and scheduling problem in the BSN. We then propose an optimization solution using the Markov decision process framework and a reinforcement learning algorithm based on the linear function approximation method, with the aim of finding the optimal data collection policy for the gateway. Through simulation results, we not only show the efficiency of the proposed solution compared with other baseline policies, but also present the analysis for data admission control policy under different classes of sensors as well as different types of data.

Embedding Based on Function Approximation for Large Scale Image Search.

The objective of this paper is to design an embedding method that maps local features describing an image (e.g., SIFT) to a higher dimensional representation useful for the image retrieval problem. First, motivated by the relationship between the linear approximation of a nonlinear function in high dimensional space and the state-of-the-art feature representation used in image retrieval, i.e., VLAD, we propose a new approach for the approximation. The embedded vectors resulted by the function approximation process are then aggregated to form a single representation for image retrieval. Second, in order to make the proposed embedding method applicable to large scale problem, we further derive its fast version in which the embedded vectors can be efficiently computed, i.e., in the closed-form. We compare the proposed embedding methods with the state of the art in the context of image search under various settings: when the images are represented by medium length vectors, short vectors, or binary vectors. The experimental results show that the proposed embedding methods outperform existing the state of the art on the standard public image retrieval benchmarks.

Fast-Tracking Stationary MOMDPs for Adaptive Management Problems

Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as ’learning by doing’, this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustainability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.

Приближение кусочно-линейных функций дискретными суммами Фурье

Приближение кусочно-линейных функций дискретными суммами Фурье

A method to solve separable fuzzy nonlinear programming problem

This paper formulates piecewise fuzzy linear approximation of fuzzy nonlinear function of a single variable. First, we construct the equation of fuzzy line segment through two fuzzy points using -cuts of two fuzzy points. Now the fuzzy nonlinear function is represented by a set of connected fuzzy line segments. Fuzzy line segments have been defined by partitioning the domain of a fuzzy nonlinear function. Then formulation of approximating fuzzy linear programming problem has been given to find the solution of fuzzy nonlinear separable programming problem. An illustrative example is given to demonstrate the methodology.

Komparativna analiza uticaja tečenja betona spregnute grede čelik - beton modelima zasnovanim na integralnoj jednačini Volterra

The paper presents analysis of the stress-strain behaviour and deflection changes due to creep in statically determinate composite steel-concrete beam according to EUROCODE 2, ACI209R-92 and Gardner&Lockman models. The mathematical model involves the equation of equilibrium, compatibility and constitutive relationship, i.e. an elastic law for the steel part and an integral-type creep law of Boltzmann - Volterra for the concrete part considering the above mentioned models. On the basis of the theory of viscoelastic body of Maslov-Arutyunian-Trost-Zerna-Bažant for determining the redistribution of stresses in beam section between concrete plate and steel beam with respect to time 't', two independent Volterra integral equations of the second kind have been derived. Numerical method based on linear approximation of the singular kernel function in the integral equation is presented. Example with the model proposed is investigated.

Improved fault detection employing hybrid memetic fuzzy modeling and adaptive filters

We propose an improved fault detection (FD) scheme based on residual signals extracted on-line from system models identified from high-dimensional measurement data recorded in multi-sensor networks. The system models are designed for an all-coverage approach and comprise linear and non-linear approximation functions representing the interrelations and dependencies among the measurement variables. The residuals obtained by comparing observed versus predicted values (i.e., the predictions achieved by the system models) are normalized subject to the uncertainty of the models and are supervised by an incrementally adaptive statistical tolerance band. Upon violation of this tolerance band, a fault alarm is triggered. The improved FD methods comes with two the main novelty aspects: (1) the development of an enhanced optimization scheme for fuzzy systems training which builds upon the SparseFIS (Sparse Fuzzy Inference Systems) approach and enhances it by embedding genetic operators for escaping local minima⟶a hybrid memetic (sparse) fuzzy modeling approach, termed as GenSparseFIS. (2) The design and application of adaptive filters on the residual signals, over time, in a sliding-window based incremental/decremental manner to smoothen the signals and to reduce the false positive rates. This gives us the freedom to tighten the tolerance band and thus to increase fault detection rates by holding the same level of false positives. In the results section, we verify that this increase is statistically significant in the case of adaptive filters when applying the proposed concepts onto four real-world scenarios (three different ones from rolling mills, one from engine test benches). The hybridization of sparse fuzzy inference systems with genetic algorithms led to the generation of more high quality models that can in turn be used in the FD process as residual generators. The new hybrid sparse memetic modeling approach also achieved fuzzy systems leading to higher fault detection rates for some scenarios.

Ocean mixing beneath Pine Island Glacier ice shelf, West Antarctica

Ice shelves around Antarctica are vulnerable to an increase in ocean-driven melting, with the melt rate depending on ocean temperature and the strength of flow inside the ice-shelf cavities. We present measurements of velocity, temperature, salinity, turbulent kinetic energy dissipation rate, and thermal variance dissipation rate beneath Pine Island Glacier ice shelf, West Antarctica. These measurements were obtained by CTD, ADCP, and turbulence sensors mounted on an Autonomous Underwater Vehicle (AUV). The highest turbulent kinetic energy dissipation rate is found near the grounding line. The thermal variance dissipation rate increases closer to the ice-shelf base, with a maximum value found ∼0.5 m away from the ice. The measurements of turbulent kinetic energy dissipation rate near the ice are used to estimate basal melting of the ice shelf. The dissipation-rate-based melt rate estimates is sensitive to the stability correction parameter in the linear approximation of universal function of the Monin-Obukhov similarity theory for stratified boundary layers. We argue that our estimates of basal melting from dissipation rates are within a range of previous estimates of basal melting.

The linearization of primary energy flow parameters change function Franklin discrete functions

The discrete transformation of Franklin is presented and the transformation on a finite interval with operations over arguments in modulus is suggested in the article. The example of determining the amplitudes of the basis functions is given. It is noted that the linear approximation of the primary energy flow parameter change function with Franklin functions enhances the energy level, which can be selected from a renewable source. Referense 6, Figures 2.

On the Best Linear Approximation of Holomorphic Functions

Let Ω be an open subset of the complex plane ℂ and let E be a compact subset of Ω. The present survey is concerned with linear n-widths for the class H ∞ (Ω) in the space C(E) and some problems on the best linear approximation of classes of Hardy–Sobolev-type in L p -spaces. It is known that the partial sums of the Faber series give the classical method for approximation of functions f ∈ H ∞ (Ω) in the metric of C(E) when E is a bounded continuum with simply connected complement and Ω is a canonical neighborhood of E. Generalizations of the Faber series are defined for the case where Ω is a multiply connected domain or a disjoint union of several such domains, while E can be split into a finite number of continua. The exact values of n-widths and asymptotic formulas for the e-entropy of classes of holomorphic functions with bounded fractional derivatives in domains of tube type are presented. Also, some results about Faber’s approximations in connection with their applications in numerical analysis are mentioned.

From free energy to expected energy: Improving energy-based value function approximation in reinforcement learning

Free-energy based reinforcement learning (FERL) was proposed for learning in high-dimensional state and action spaces. However, the FERL method does only really work well with binary, or close to binary, state input, where the number of active states is fewer than the number of non-active states. In the FERL method, the value function is approximated by the negative free energy of a restricted Boltzmann machine (RBM). In our earlier study, we demonstrated that the performance and the robustness of the FERL method can be improved by scaling the free energy by a constant that is related to the size of network. In this study, we propose that RBM function approximation can be further improved by approximating the value function by the negative expected energy (EERL), instead of the negative free energy, as well as being able to handle continuous state input. We validate our proposed method by demonstrating that EERL: (1) outperforms FERL, as well as standard neural network and linear function approximation, for three versions of a gridworld task with high-dimensional image state input; (2) achieves new state-of-the-art results in stochastic SZ-Tetris in both model-free and model-based learning settings; and (3) significantly outperforms FERL and standard neural network function approximation for a robot navigation task with raw and noisy RGB images as state input and a large number of actions.

Least squares approximate policy iteration for learning bid prices in choice-based revenue management

We consider the revenue management problem of capacity control under customer choice behavior. An exact solution of the underlying stochastic dynamic program is difficult because of the multi-dimensional state space and, thus, approximate dynamic programming (ADP) techniques are widely used. The key idea of ADP is to encode the multi-dimensional state space by a small number of basis functions, often leading to a parametric approximation of the dynamic program’s value function. In general, two classes of ADP techniques for learning value function approximations exist: mathematical programming and simulation. So far, the literature on capacity control largely focuses on the first class.In this paper, we develop a least squares approximate policy iteration (API) approach which belongs to the second class. Thereby, we suggest value function approximations that are linear in the parameters, and we estimate the parameters via linear least squares regression. Exploiting both exact and heuristic knowledge from the value function, we enforce structural constraints on the parameters to facilitate learning a good policy. We perform an extensive simulation study to investigate the performance of our approach. The results show that it is able to obtain competitive revenues compared to and often outperforms state-of-the-art capacity control methods in reasonable computational time. Depending on the scarcity of capacity and the point in time, revenue improvements of around 1% or more can be observed. Furthermore, the proposed approach contributes to simulation-based ADP, bringing forth research on numerically estimating piecewise linear value function approximations and their application in revenue management environments.

Computing tight bounds via piecewise linear functions through the example of circle cutting problems

This paper discusses approximations of continuous and mixed-integer non-linear optimization problems via piecewise linear functions. Various variants of circle cutting problems are considered, where the non-overlap of circles impose a non-convex feasible region. While the paper is written in an “easy-to-understand” and “hands-on” style which should be accessible to graduate students, also new ideas are presented. Specifically, piecewise linear functions are employed to yield mixed-integer linear programming problems which provide lower and upper bounds on the original problem, the circle cutting problem. The piecewise linear functions are modeled by five different formulations, containing the incremental and logarithmic formulations. Another variant of the cutting problem involves the assignment of circles to pre-defined rectangles. We introduce a new global optimization algorithm, based on piecewise linear function approximations, which converges in finitely many iterations to a globally optimal solution. The discussed formulations are implemented in GAMS. All GAMS-files are available for download in the Electronic supplementary material. Extensive computational results are presented with various illustrations.

Finite element analysis of deformation in early stage of multi-pass circumferential dissimilar welding of thick-walled pipes with narrow gap

Three-dimensional thermal elastic-plastic finite element analyses were conducted in order to reveal the mechanism of gap shrinkage in early stage of multi-pass narrow gap welding of thick-walled dissimilar pipes and thick plates. The gap shrinkage up to the 8th pass indicates that the shrinkage of plates becomes to be much larger than that of pipes with increasing the weld passes. In addition, through the decomposition of gap shrinkage to transverse shrinkage and angular distortion, it is found that the gap shrinkage of pipes is mainly governed by the transverse shrinkage, while the shrinkage of plates is influenced by both the transverse shrinkage and angular distortion. Moreover, the serial computational results with various groove shapes suggest that the transverse shrinkage of pipes almost linearly increases with increasing the weld pass, and its increment can be predicted by a linear approximation function obtained by the transverse shrinkage of plates regardless of the groove shape.

Modified Delaunay empty sphere condition in the problem of approximation of the gradient

The classical Schwarz example shows that piecewise- linear approximation of smooth functions does not necessary yield convergence of the derivatives. However, in the planar case, the required convergence holds if the triangulation of the grid satisfies the empty sphere condition (that is, it is a Delaunay triangulation). These results do not extend to the multidimensional case, as is shown by our published examples. We give a modified empty sphere condition that also guarantees the necessary approximation in the multidimensional case.

Efficient Average Reward Reinforcement Learning Using Constant Shifting Values

There are two classes of average reward reinforcement learning (RL) algorithms: model-based ones that explicitly maintain MDP models and model-free ones that do not learn such models. Though model-free algorithms are known to be more efficient, they often cannot converge to optimal policies due to the perturbation of parameters. In this paper, a novel model-free algorithm is proposed, which makes use of constant shifting values (CSVs) estimated from prior knowledge. To encourage exploration during the learning process, the algorithm constantly subtracts the CSV from the rewards. A terminating condition is proposed to handle the unboundedness of Q-values caused by such substraction. The convergence of the proposed algorithm is proved under very mild assumptions. Furthermore, linear function approximation is investigated to generalize our method to handle large-scale tasks. Extensive experiments on representative MDPs and the popular game Tetris show that the proposed algorithms significantly outperform the state-of-the-art ones.