Linear Function Approximation Research Articles

Feature subset selection for logistic regression via mixed integer optimization

This paper concerns a method of selecting a subset of features for a logistic regression model. Information criteria, such as the Akaike information criterion and Bayesian information criterion, are employed as a goodness-of-fit measure. The purpose of our work is to establish a computational framework for selecting a subset of features with an optimality guarantee. For this purpose, we devise mixed integer optimization formulations for feature subset selection in logistic regression. Specifically, we pose the problem as a mixed integer linear optimization problem, which can be solved with standard mixed integer optimization software, by making a piecewise linear approximation of the logistic loss function. The computational results demonstrate that when the number of candidate features was less than 40, our method successfully provided a feature subset that was sufficiently close to an optimal one in a reasonable amount of time. Furthermore, even if there were more candidate features, our method often found a better subset of features than the stepwise methods did in terms of information criteria.

Actor-Critic Algorithms with Online Feature Adaptation

We develop two new online actor-critic control algorithms with adaptive feature tuning for Markov Decision Processes (MDPs). One of our algorithms is proposed for the long-run average cost objective, while the other works for discounted cost MDPs. Our actor-critic architecture incorporates parameterization both in the policy and the value function. A gradient search in the policy parameters is performed to improve the performance of the actor. The computation of the aforementioned gradient, however, requires an estimate of the value function of the policy corresponding to the current actor parameter. The value function, on the other hand, is approximated using linear function approximation and obtained from the critic. The error in approximation of the value function, however, results in suboptimal policies. In our article, we also update the features by performing a gradient descent on the Grassmannian of features to minimize a mean square Bellman error objective in order to find the best features. The aim is to obtain a good approximation of the value function and thereby ensure convergence of the actor to locally optimal policies. In order to estimate the gradient of the objective in the case of the average cost criterion, we utilize the policy gradient theorem, while in the case of the discounted cost objective, we utilize the simultaneous perturbation stochastic approximation (SPSA) scheme. We prove that our actor-critic algorithms converge to locally optimal policies. Experiments on two different settings show performance improvements resulting from our feature adaptation scheme.

Efficient Actor-Critic Algorithm with Hierarchical Model Learning and Planning.

To improve the convergence rate and the sample efficiency, two efficient learning methods AC-HMLP and RAC-HMLP (AC-HMLP with ℓ 2-regularization) are proposed by combining actor-critic algorithm with hierarchical model learning and planning. The hierarchical models consisting of the local and the global models, which are learned at the same time during learning of the value function and the policy, are approximated by local linear regression (LLR) and linear function approximation (LFA), respectively. Both the local model and the global model are applied to generate samples for planning; the former is used only if the state-prediction error does not surpass the threshold at each time step, while the latter is utilized at the end of each episode. The purpose of taking both models is to improve the sample efficiency and accelerate the convergence rate of the whole algorithm through fully utilizing the local and global information. Experimentally, AC-HMLP and RAC-HMLP are compared with three representative algorithms on two Reinforcement Learning (RL) benchmark problems. The results demonstrate that they perform best in terms of convergence rate and sample efficiency.

Learning the variance of the reward-to-go

In Markov decision processes (MDPs), the variance of the reward-to-go is a natural measure of uncertainty about the long term performance of a policy, and is important in domains such as finance, resource allocation, and process control. Currently however, there is no tractable procedure for calculating it in large scale MDPs. This is in contrast to the case of the expected reward-to-go, also known as the value function, for which effective simulation-based algorithms are known, and have been used successfully in various domains. In this paper we extend temporal difference (TD) learning algorithms to estimating the variance of the reward-to-go for a fixed policy. We propose variants of both TD(0) and LSTD(λ) with linear function approximation, prove their convergence, and demonstrate their utility in an option pricing problem. Our results show a dramatic improvement in terms of sample efficiency over standard Monte-Carlo methods, which are currently the state-of-the-art.

A Novel Probabilistic Optimal Power Flow Model With Uncertain Wind Power Generation Described by Customized Gaussian Mixture Model

A novel probabilistic optimal power flow (P-OPF) model with chance constraints that considers the uncertainties of wind power generation (WPG) and load is proposed in this paper. An affine generation dispatch strategy is adopted to balance the system power uncertainty by several conventional generators, and thus the linear approximation of the cost function with respect to the power uncertainty is proposed to compute the quantile (which is also recognized as the value-at-risk) corresponding to a given probability value. The proposed model applies this quantile as the objective function and minimizes it to meet distinct probabilistic cost regulation purposes via properly selecting the given probability. In particular, the hedging effect due to the used affine generation dispatch is also thoroughly investigated. In addition, an analytical method to calculate probabilistic load flow (PLF) is developed with the probability density function of WPG, which is proposed to be approximated by a customized Gaussian mixture model whose parameters are easily obtained. Accordingly, it is successful to analytically compute the chance constraints on the transmission line power and the power outputs of conventional units. Numerical studies of two benchmark systems show the satisfactory accuracy of the PLF method, and the effectiveness of the proposed P-OPF model.

Numerical Solution of a Nonlinear Integro-Differential Equation

An algorithm for the numerical solution of a nonlinear integro-differential equation arising in the single-species annihilation reaction $A + A \rightarrow\varnothing$ modeling is discussed. Finite difference method together with the linear approximation of the unknown function is considered. For divergent integrals presented in the equation for dimension $d=2$ a regularization is used. Some numerical results are presented.

An Improved FastSLAM Algorithm Based on Revised Genetic Resampling and SR-UPF

FastSLAM is a popular framework which uses a Rao-Blackwellized particle filter to solve the simultaneous localization and mapping problem (SLAM). However, in this framework there are two important potential limitations, the particle depletion problem and the linear approximations of the nonlinear functions. To overcome these two drawbacks, this paper proposes a new FastSLAM algorithm based on revised genetic resampling and square root unscented particle filter (SR-UPF). Double roulette wheels as the selection operator, and fast Metropolis-Hastings (MH) as the mutation operator and traditional crossover are combined to form a new resampling method. Amending the particle degeneracy and keeping the particle diversity are both taken into considerations in this method. As SR-UPF propagates the sigma points through the true nonlinearity, it decreases the linearization errors. By directly transferring the square root of the state covariance matrix, SR-UPF has better numerical stability. Both simulation and experimental results demonstrate that the proposed algorithm can improve the diversity of particles, and perform well on estimation accuracy and consistency.

Development of Galerkin Finite Element Method Three-dimensional Computational Code for the Multigroup Neutron Diffusion Equation with Unstructured Tetrahedron Elements

Abstract In the present paper, development of the three-dimensional (3D) computational code based on Galerkin finite element method (GFEM) for solving the multigroup forward/adjoint diffusion equation in both rectangular and hexagonal geometries is reported. Linear approximation of shape functions in the GFEM with unstructured tetrahedron elements is used in the calculation. Both criticality and fixed source calculations may be performed using the developed GFEM-3D computational code. An acceptable level of accuracy at a low computational cost is the main advantage of applying the unstructured tetrahedron elements. The unstructured tetrahedron elements generated with Gambit software are used in the GFEM-3D computational code through a developed interface. The forward/adjoint multiplication factor, forward/adjoint flux distribution, and power distribution in the reactor core are calculated using the power iteration method. Criticality calculations are benchmarked against the valid solution of the neutron diffusion equation for International Atomic Energy Agency (IAEA)-3D and Water-Water Energetic Reactor (VVER)-1000 reactor cores. In addition, validation of the calculations against the P 1 approximation of the transport theory is investigated in relation to the liquid metal fast breeder reactor benchmark problem. The neutron fixed source calculations are benchmarked through a comparison with the results obtained from similar computational codes. Finally, an analysis of the sensitivity of calculations to the number of elements is performed.

Optimal Piecewise Linear Function Approximation for GPU-Based Applications

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of these kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the tradeoff between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern graphics processing units, where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions; we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

Hyperbolic cross approximation in infinite dimensions

We give tight upper and lower bounds of the cardinality of the index sets of certain hyperbolic crosses which reflect mixed Sobolev–Korobov-type smoothness and mixed Sobolev-analytic-type smoothness in the infinite-dimensional case where specific summability properties of the smoothness indices are fulfilled. These estimates are then applied to the linear approximation of functions from the associated spaces in terms of the ε-dimension of their unit balls. Here, the approximation is based on linear information. Such function spaces appear for example for the solution of parametric and stochastic PDEs. The obtained upper and lower bounds of the approximation error as well as of the associated ε-complexities are completely independent of any parametric or stochastic dimension. Moreover, the rates are independent of the parameters which define the smoothness properties of the infinite-variate parametric or stochastic part of the solution. These parameters are only contained in the order constants. This way, linear approximation theory becomes possible in the infinite-dimensional case and corresponding infinite-dimensional problems get tractable.

A Conjugate Gradient Algorithm with Function Value Information and N-Step Quadratic Convergence for Unconstrained Optimization.

It is generally acknowledged that the conjugate gradient (CG) method achieves global convergence—with at most a linear convergence rate—because CG formulas are generated by linear approximations of the objective functions. The quadratically convergent results are very limited. We introduce a new PRP method in which the restart strategy is also used. Moreover, the method we developed includes not only n-step quadratic convergence but also both the function value information and gradient value information. In this paper, we will show that the new PRP method (with either the Armijo line search or the Wolfe line search) is both linearly and quadratically convergent. The numerical experiments demonstrate that the new PRP algorithm is competitive with the normal CG method.

Multiscale Q-learning with linear function approximation

We present in this article a two-timescale variant of Q-learning with linear function approximation. Both Q-values and policies are assumed to be parameterized with the policy parameter updated on a faster timescale as compared to the Q-value parameter. This timescale separation is seen to result in significantly improved numerical performance of the proposed algorithm over Q-learning. We show that the proposed algorithm converges almost surely to a closed connected internally chain transitive invariant set of an associated differential inclusion.

Stabilized Implicit Cosimulation Method: Solver Coupling With Algebraic Constraints for Multibody Systems

In this manuscript, an implicit cosimulation method is analyzed, where the solvers are coupled by algebraic constraint equations. We discuss cosimulation approaches on index-2 and on index-1 level and investigate constant, linear and quadratic approximation functions for the coupling variables. The key idea of the method presented here is to discretize the Lagrange multipliers between the macrotime points (extended multiplier approach) so that the coupling equations and their time derivatives can simultaneously be fulfilled at the macrotime points. Stability and convergence of the method are investigated in detail. Following the stability analysis for time integration schemes based on Dahlquist's test equation, an appropriate cosimulation test model is used to examine the numerical stability of the presented cosimulation method. Discretizing the cosimulation test model by means of a linear cosimulation approach yields a system of linear recurrence equations. The spectral radius of the recurrence equation system characterizes the numerical stability of the underlying cosimulation method. As for time integration methods, 2D stability plots are used to graphically illustrate the stability behavior of the coupling approach.

Who's Next? Scheduling Personalization Services with Variable Service Times

Online personalization has become quite prevalent in recent years, with firms able to derive additional profits from such services. As the adoption of such services grows, firms implementing such practices face some operational challenges. One important challenge lies in the complexity associated with the personalization process and how to deploy available resources to handle such complexity. The complexity is exacerbated when a site faces a large volume of requests in a short amount of time, as is often the case for e-commerce and content delivery sites. In such situations, it is generally not possible for a site to provide perfectly personalized service to all requests. Instead, a firm can provide differentiated service to requests based on the amount of profiling information available about the visitor. We consider a scenario where the revenue function is concave, capturing the diminishing returns from personalization effort. Using a batching approach, we determine the optimal scheduling policy (i.e., time allocation and sequence of service) for a batch that accounts for the externality cost incurred when a request is provided service before other waiting requests. The batching approach leads to sunk costs incurred when visitors wait for the next batch to begin. An optimal admission control policy is developed to prescreen new request arrivals. We show how the policy can be implemented efficiently when the revenue function is complex and there are a large number of requests that can be served in a batch. Numerical experiments show that the proposed approach leads to substantial improvements over a linear approximation of the concave revenue function. Interestingly, we find that the improvements in firm profits are not only (or primarily) due to the different service times that are obtained when using the nonlinear personalization function—there is a ripple effect on the admission control policy that incorporates these optimized service times, which contributes even more to the additional profits than the service time optimization by itself.

Novel design solutions for fishing reel mechanisms

Currently, there are various reels on the market regarding the type of mechanism, which achieves the winding and unwinding of the line. The designers have the purpose of obtaining a linear transmission function, by means of a simple and small-sized mechanism. However, the present solutions are not satisfactory because of large deviations from linearity of the transmission function and complexity of mechanical schema. A novel solution for the reel spool mechanism is proposed. Its kinematic schema and synthesis method are described. The kinematic schema of the chosen mechanism is based on a noncircular gear in series with a scotch-yoke mechanism. The yoke is driven by a stud fixed on the driving noncircular gear. The drawbacks of other models regarding the effects occurring at the ends of the spool are eliminated through achieving an appropriate transmission function of the spool. The linear function approximation with curved end-arches appropriately computed to ensure mathematical continuity is very good. The experimental results on the mechanism model validate the theoretical approach. The developed mechanism solution is recorded under a reel spool mechanism patent.

TH‐AB‐BRB‐08: Optimizing Global Liver Function in Liver SBRT Treatment Planning

Purpose:Liver SBRT patients have variable pre‐treatment liver function, which strongly influences liver toxicity risk. Previous studies include a “perfusion‐weighted” mean dose model as a surrogate for liver function in plan optimization. This work investigates the benefit of using a more sophisticated dose‐response model that incorporates two important dose thresholds: (i) based on pre‐treatment function, a significant low‐dose may need to be exceeded before noticeable damage is done, and (ii) a high‐dose saturation point, beyond which no additional damage is done.Methods:Voxel‐based liver perfusion from DCE‐MRI was obtained. Two optimization models subject to the same linear dose constraints (e.g., minimum target EUD, maximum critical structure dose) were compared: the original linear model, which use pre‐treatment perfusion‐weighted mean liver dose only, and a new model designed to directly optimize the predicted post‐treatment global liver function. The latter, a highly non‐linear non‐convex problem, was approximately solved by optimizing a piece‐wise linear approximation of the objective function with a customized mixed‐integer linear programming (MILP) based algorithm. 2D synthetic and 3D clinical cases were studied to assess the potential benefit of the new model.Results:Compared to those of the original model, dose distributions for the functional dose response model take advantage of voxels with saturated damage to deliver the required dose to the target instead of inflicting additional damage on other well‐functioning voxels. This leads to an improvement in predicted post‐treatment global liver function. The customized solution method also improves optimization efficiency compared to solving the MILP conventionally.Conclusion:Functional imaging such as DCE‐MRI can be used during treatment planning to maximize potential post‐treatment function. However, simple approximations such as minimization of perfusion‐weighted mean dose may not be sufficient. Alternatively, a dose‐response model that reflects response threshold and saturation effects was shown to improve the prediction of post‐treatment function in liver SBRT.Supported by P01 CA 059827

The use of SHS-process slag for the preparation of foundry sand

The article presents the construction of a linear approximation of the response function (strength of molding sand on tensile strength and air permeability) in a given area by changing the investigated factors for which charge makeup of SHS-cast iron obtaining process and content of the slag product of this process in the molding mixture were used. The analysis of regression coefficients significance of the objective function was done to identify the most important input parameters.

Linear approximation of vector functions

In a problem on the approximation of a vector function continuous on an interval by linear functions in the Chebyshev metric, necessary and sufficient conditions on the best approximation function are established.

Circuit Implementations, Bifurcations and Chaos of a Novel Fractional-Order Dynamical System**Supported by the National Natural Science Foundation of China under Grant No 51475246, the Natural Science Foundation of Jiangsu Province under Grant No Bk20131402, and the Ministry-of-Education Overseas Returnees Start-up Research Fund under Grant No [2012]1707.

Linear transfer function approximations of the fractional integrators 1/sm with m = 0.80–0.99 with steps of 0.01 are calculated systemically from the fractional order calculus and frequency-domain approximation method. To illustrate the effectiveness for fractional functions, the magnitude Bode diagrams of the actual and approximate transfer functions 1/sm with a slope of —20m dB/decade are depicted. By using the transfer function approximations of the fractional integrators, a new fractional-order nonlinear system is investigated through the bifurcation diagram and Lyapunov exponent. The corresponding circuit of the fractional-order system is designed and the experimental results match perfectly with the numerical simulations.

A Generalized Reduced Linear Program for Markov Decision Processes

Markov decision processes (MDPs) with large number of states are of high practical interest. However, conventional algorithms to solve MDP are computationally infeasible in this scenario. Approximate dynamic programming (ADP) methods tackle this issue by computing approximate solutions. A widely applied ADP method is approximate linear program (ALP) which makes use of linear function approximation and offers theoretical performance guarantees. Nevertheless, the ALP is difficult to solve due to the presence of a large number of constraints and in practice, a reduced linear program (RLP) is solved instead. The RLP has a tractable number of constraints sampled from the original constraints of the ALP. Though the RLP is known to perform well in experiments, theoretical guarantees are available only for a specific RLP obtained under idealized assumptions. In this paper, we generalize the RLP to define a generalized reduced linear program (GRLP) which has a tractable number of constraints that are obtained as positive linear combinations of the original constraints of the ALP. The main contribution of this paper is the novel theoretical framework developed to obtain error bounds for any given GRLP. Central to our framework are two max-norm contraction operators. Our result theoretically justifies linear approximation of constraints. We discuss the implication of our results in the contexts of ADP and reinforcement learning. We also demonstrate via an example in the domain of controlled queues that the experiments conform to the theory.