Linear Value Function Research Articles

Quadratic equations with absolute values: an example of developing proof in mathematics students

Proof is the most important and fundamental aspect of mathematics. However, it is generally agreed by mathematics teachers at upper high school and first year university that students lack the ideas of proof. How to develop this in students is a common, difficult and perennial problem. In this paper, we develop some ideas of proof based on the number of solutions of an equation involving a quadratic polynomial and absolute values of linear functions. We begin with the simple idea of making more precise a question that is posed, investigating the problem through exploration, and arriving at some conjectures. We proceed to determine ways to prove the conjectures and thus convert them into theorems. The question discussed in this paper arose in a session for high school students held at the Department of Mathematics and Statistics of the University of Western Australia. The result is an excellent and interesting illustration of problem posing and solving in mathematics that underpins mathematical thinking. The paper is accessible to final year high school and first-year university mathematics students. We expect that it will serve as a resource for and inspire further ideas and examples for mathematics teachers, for teaching proof to students.

Decision-making in the multiphase optimization strategy: Applying decision analysis for intervention value efficiency to optimize an information leaflet to promote key antecedents of medication adherence.

Advances in the multiphase optimization strategy (MOST) have suggested a new approach, decision analysis for intervention value efficiency (DAIVE), for selecting an optimized intervention based on the results of a factorial optimization trial. The new approach opens possibilities to select optimized interventions based on multiple valued outcomes. We applied DAIVE to identify an optimized information leaflet intended to support eventual adherence to adjuvant endocrine therapy for women with breast cancer. We used empirical performance data for five candidate leaflet components on three hypothesized antecedents of adherence: beliefs about the medication, objective knowledge about AET, and satisfaction with medication information. Using data from a 25 factorial trial (n = 1603), we applied the following steps: (i) We used Bayesian factorial analysis of variance to estimate main and interaction effects for the five factors on the three outcomes. (ii) We used posterior distributions for main and interaction effects to estimate expected outcomes for each leaflet version (32 total). (iii) We scaled and combined outcomes using a linear value function with predetermined weights indicating the relative importance of outcomes. (iv) We identified the leaflet that maximized the value function as the optimized leaflet, and we systematically varied outcome weights to explore robustness. The optimized leaflet included two candidate components, side-effects, and patient input, set to their higher levels. Selection was generally robust to weight variations consistent with the initial preferences for three outcomes. DAIVE enables selection of optimized interventions with the best-expected performance on multiple outcomes.

An approximated dynamic programming model for the supply vessel fleet sizing problem

The offshore oil and gas exploration and production activities heavily rely on a fleet of supply vessels, which play a crucial role in regularly providing necessary provisions from an onshore supply depot to the offshore installations. For Petrobras, the decision-making process concerning the number of supply vessels to be hired for the upcoming two years holds significant economic implications, given that the daily rates for a single supply vessel can soar to tens of thousands of USDs.In this study, we address the challenges of this heterogeneous fleet sizing problem by presenting an innovative approximated dynamic programming (ADP) model. The model takes into account the uncertainties associated with sea conditions and future demand, making it well-suited for real-world applications. To tackle this problem, we explored two types of approximate value functions: a piecewise linear concave value function and a neural network value function. Both methods underwent rigorous testing and demonstrated their capability to effectively address real-world problems. Consequently, our proposed method exhibits promising potential for practical implementation, allowing companies like Petrobras to accurately assess and determine the ideal fleet composition. By introducing this ADP model, we aim to provide valuable insights and solutions that enhance decision-making processes related to supply vessel fleet sizing in offshore operations. Through considering various uncertainties and complexities, our approach offers a robust and reliable framework to support critical economic decisions in this dynamic industry.

Approximate dynamic programming approach to efficient metro train timetabling and passenger flow control strategy with stop-skipping

Urban metro systems continuously face high travel demand during rush hours, which brings excessive energy waste and high risk to passengers. In order to alleviate passenger congestion, improve train service levels and reduce energy consumption, a nonlinear dynamic programming (DP) model of efficient metro train timetabling and passenger flow control strategy with stop-skipping is presented, which consists of state transition equations concerning train traffic and passenger load. To overcome the curse of dimensionality, the formulated nonlinear DP problem is transformed into a discrete Markov decision process, and a novel approximate dynamic programming (ADP) approach is designed based on the lookahead policy and linear parametric value function approximation. Finally, the effectiveness of this method is verified by three groups of numerical experiments. Compared with Particle Swarm Optimization (PSO) and Simulated Annealing (SA), the designed ADP approach could obtain high-quality solutions quickly, which makes it applicable to the practical implementation of metro operations.

A Bayesian model for multicriteria sorting problems

Decision makers are often interested in assigning alternatives to preference classes under multiple criteria instead of choosing the best alternative or ranking all the alternatives. Firms need to categorize suppliers based on performance, credit agencies need to classify customers according to their risks, and graduate programs need to decide who to admit. In this article, we develop an interactive Bayesian algorithm to aid a decision maker (DM) with a multicriteria sorting problem by learning about her preferences and using that knowledge to sort alternatives. We assume the DM has a linear value function and value thresholds for preference classes. Our method specifies an informative prior distribution on the uncertain parameters. At each stage of the process, we compare the expected cost of stopping with the expected cost of continuing to consult the DM. If it is optimal to continue, we select an alternative to present to the DM and, given the DM’s response, we update the prior distribution using Bayes’ Theorem. The goal of the algorithm is to minimize expected total cost. We develop lower bounds on the optimal cost and study the performance of a heuristic policy that presents the DM alternatives with the highest expected cost of misplacement.

A bi-criteria moving-target travelling salesman problem under uncertainty

This article concerns a variant of moving target travelling salesman problem where the number and locations of targets vary with time and realizations of random trajectories. Managerial objectives are to maximize the number of visits to different targets and to minimize the total travel distance. Employing a linear value function for finding supported Pareto-efficient solutions, we develop a two-stage stochastic programming model. We propose an iterative randomized dynamic programming (RDP) algorithm which converges to a global optimum with probability one. Each iteration in RDP involves a randomized backward and forward recursion stage as well as options for improving any given schedule: swaps of targets and optimization of timing for visits. An integer linear programming (ILP) model is developed and solved by a standard ILP solver to evaluate the performance of RDP on instances of real data for scheduling an environmental surveillance boat to visit ships navigating in the Baltic Sea. Due to a huge number of binary variables, the ILP model in practice becomes intractable. For small to medium size data sets, the Pareto-efficiency of solutions found by RDP and ILP solver are equal within a reasonable tolerance; however, RDP is significantly faster and able to deal with large-scale problems in practice.

A Behavior Decision Method Based on Reinforcement Learning for Autonomous Driving

Autonomous driving vehicles can reduce congestion and improve safety while increasing traffic efficiency. To reflect the quality of driving more comprehensively, the driving safety, efficiency, and occupant comfort should be jointly optimized for autonomous vehicles. Furthermore, in order to cope with complicated traffic environments and achieve satisfactory driving performance, a powerful behavior decision-making module is indispensable for autonomous vehicles. Toward this end, we study a reinforcement-learning (RL)-based method to intelligently make the behavior decision in this article. A Markov decision process (MDP) model is first formulated with a comprehensive reward function, including the effects of driving safety, efficiency, and comfort. The knowledge of the surrounding vehicles is also leveraged to exploit the behavior prediction of the target vehicle. We then propose a behavior decision strategy based on the actor–critic (AC) mechanism, which can efficiently learn both a Gaussian policy function and a linear value function. Finally, the real traffic data are used to build up the simulations for evaluating the performances of the proposed method thoroughly. Simulation results show that our proposed method can significantly reduce the collision rate for autonomous vehicles.

Efficient compact linear programs for network revenue management

We are concerned with computing bid prices in network revenue management using approximate linear programming. It is well-known that affine value function approximations yield bid prices which are not sensitive to remaining capacity. The analytic reduction to compact linear programs allows the efficient computation of such bid prices. On the other hand, capacity-dependent bid prices can be obtained using separable piecewise linear value function approximations. Even though compact linear programs have been derived for this case also, they are still computationally much more expensive compared to using affine functions. We propose compact linear programs requiring substantially smaller computing times while, simultaneously, significantly improving the performance of capacity-independent bid prices. This simplification is achieved by taking into account remaining capacity only if it becomes scarce. Although our proposed linear programs are relaxations of the unreduced approximate linear programs, we conjecture equivalence and provide according numerical support. We measure the quality of an approximation by the difference between the expected performance of an induced policy and the corresponding theoretical upper bound. Using this paradigm in numerical experiments, we demonstrate the competitiveness of our proposed linear programs.

Dynamic Valuation of Battery Lifetime

This paper proposes a dynamic valuation framework to determine the opportunity value of battery capacity degradation in grid applications based on the internal degradation mechanism and utilization scenarios. The proposed framework follows a dynamic programming approach and includes a piecewise linear value function approximation solution that solves the optimization problem over a long planning horizon. The paper provides two case studies on price arbitrage and frequency regulation using real market and system data to demonstrate the broad applicability of the proposed framework. Results show that the battery lifetime value is critically dependent on both the external market environment and its internal state of health. On the grid service side, results show that second-life batteries can provide more than 50% of the value compared to new batteries, and frequency regulation provides two times more revenue than price arbitrage throughout the battery lifetime.

Optimal Use and Replenishment of Two Substitutable Raw Materials in a Stochastic Capacitated Make-to-Order Production System

Problem definition: We study a multiperiod, nonstationary, make-to-order, joint production and inventory model where two kinds of input raw materials with availability uncertainties and different output conversion rates can be blended and then processed in a production line with stochastic capacity to produce the output product. Academic/practical relevance: The problem is motivated by the practice in coal-fired power plants, an important part of the energy sector, where two types of coal with different energy content per unit mass are blended for electrical power generation. Our model is the first to capture the key operational features in this context. Methodology: We model the problem as a Markov decision process and develop a novel approximate optimization approach to analyze and characterize the structure of the optimal policy. Results: We show that a use-down-to/balancing production policy and modified order-up-to ordering policy is optimal. We also propose a heuristic policy based on piece-wise linear value function approximation. Whereas computing the value function approximation via brute-force is time-consuming because of the curse of dimensionality, we leverage the structure of the optimal policy to develop an algorithm that greatly improves the computational time of the value function approximation. Our numerical studies on both a synthetic data set and real-world data show that the proposed heuristic provides significant profit improvement over three simpler straw policies, some of which are used in practice. Managerial implications: Our paper suggests the significant profit improvement opportunity of using our proposed policy and demonstrates how one can develop computationally more efficient heuristic policies by leveraging the structure of the optimal policy.

Application of Multicriteria Decision Analysis to Determine the Value of Prophylaxis Relative to On-Demand Treatment in Hemophilia A and Emicizumab versus Replacement Therapy in the Greek Healthcare Setting.

Hemophilia A (HA) is a rare disease that is characterized by congenital underproduction or dysfunction of the endogenous coagulation factor VIII (FVIII). The aim of the present study was to determine the value of prophylaxis versus on-demand treatment strategies for moderate to severe HA (MtSHA) patients and the value of emicizumab in the prophylaxis of MtSHA in Greece, compared with short half-life (SHL) FVIII and extended half-life (EHL) FVIII through multicriteria decision analysis (MCDA). A literature review was performed to identify a set of criteria relevant to the therapeutic approaches and therapies under investigation. A performance matrix was populated by two literature reviews and meta-analyses. The criteria selected were hierarchically classified by allocating weights on a 0-100 scale. The performances of therapies were scored at the 100-point scale. The value judgments utilized for weighing and scoring were sourced via a survey among independent multidisciplinary system stakeholders. A linear additive value function was used for the calculation of total value estimates. The participants ranked 'annual number of bleedings per patient' and 'percentage of target joint bleeds' as the most important criteria, while the least important criterion was the 'annual treatment cost' for both assessments. Based on the weights elicited and the performance in each criterion, the overall value score was higher for prophylaxis treatment (58.27) compared with on-demand treatment (40.13). In the other comparison, the most valued treatment was emicizumab (77.05) followed by EHL FVIII (71.52) and SHL FVIII (19.88). According to the participants, the most important factors for managing MtSHA patients are those related to successful management of bleeding events given their contribution to improved quality of life (QoL) and reduced morbidity. This MCDA has shown that the prophylaxis strategy was perceived as a more valuable option for managing MtSHA patients when compared with the on-demand strategy. Moreover, emicizumab adds higher value and improves patient QoL compared with replacement therapy for MtSHA in Greece. Emicizumab addresses important unmet needs due to its improved efficacy and mode of administration.

A personalized decision aid for prostate cancer shared decision making

BackgroundA shared decision-making model is preferred for engaging prostate cancer patients in treatment decisions. However, the process of assessing an individual’s preferences and values is challenging and not formalized. The purpose of this study is to develop an automated decision aid for patient-centric treatment decision-making using decision analysis, preference thresholds and value elicitations to maximize the compatibility between a patient’s treatment expectations and outcome.MethodsA template for patient-centric medical decision-making was constructed. The inputs included prostate cancer risk group, pre-treatment health state, treatment alternatives (primarily focused on radiation in this model), side effects (erectile dysfunction, urinary incontinence, nocturia and bowel incontinence), and treatment success (5-year freedom from biochemical failure). A linear additive value function was used to combine the values for each attribute (side effects, success and the alternatives) into a value for all prospects. The patient-reported toxicity probabilities were derived from phase II and III trials. The probabilities are conditioned on the starting state for each of the side effects. Toxicity matrices for erectile dysfunction, urinary incontinence, nocturia and bowel incontinence were created for the treatment alternatives. Toxicity probability thresholds were obtained by identifying the patient’s maximum acceptable threshold for each of the side effects. Results are represented as a visual. R and Rstudio were used to perform analyses, and R Shiny for application creation.ResultsWe developed a web-based decision aid. Based on preliminary use of the application, every treatment alternative could be the best choice for a decision maker with a particular set of preferences. This result implies that no treatment has determinist dominance over the remaining treatments and that a preference-based approach can help patients through their decision-making process, potentially affecting compliance with treatment, tolerance of side effects and satisfaction with the decision.ConclusionsWe present a unique patient-centric prostate cancer treatment decision aid that systematically assesses and incorporates a patient’s preferences and values to rank treatment options by likelihood of achieving the preferred outcome. This application enables the practice and study of personalized medicine. This model can be expanded to include additional inputs, such as genomics, as well as competing, concurrent or sequential therapies.

Two Sides of the Same Coin? A Dual Multiple Criteria Decision Analysis of Novel Treatments Against Rheumatoid Arthritis in Physicians and Patients

Available treatment options for rheumatoid arthritis (RA) differ in important aspects. In this sense, each RA treatment option is accompanied by a spectrum of characteristics that collectively constitute its comprehensive "value," as viewed from the physician's or the patient's perspective. The objective of this study was to perform a multiple criteria decision analysis of different RA treatments from the perspective of physicians and patients and to outline the respective aspects of value for each treatment METHODS: A literature review was performed for constructing a set of criteria (N=8) for the multiple criteria decision analysis. Workshops for the elicitation of preferences occurred separately for physicians and patients. A performance matrix was populated via 2 network meta-analyses plus converged clinical opinion. Criteria were hierarchically classified by application of pairwise comparisons, and criteria weights were attributed by point allocation through convergence of opinions. Performances in both panels were scored by using a 100-point scale. A linear additive value function was used for the calculation of total value estimates. Both panels provided their consensus. The hierarchical classification of attributes from the physician perspective placed the highest values on the criteria of severe adverse events, clinical efficacy, route of administration, and cost per year for the third-party payer. From the patient perspective, the highest ranking criteria were clinical efficacy, severe adverse events, percentage of patients remaining with the same targeted immune modulator for 1 year ("drug survival"), and cost per year for the third-party payer. In an era of multiple options and varying preferences, RA treatments must be evaluated by taking into consideration patients' preferences as well, as to cover the full spectrum of value elements rather than simply clinical outcomes. The results of this analysis show that physicians and patients share similarities but also marked differences in terms of the aspects of treatment that they perceive as more valuable.

Fast Feature Selection for Linear Value Function Approximation

Linear value function approximation is a standard approach to solving reinforcement learning problems with large state spaces. Since designing good approximation features is difficult, automatic feature selection is an important research topic. We propose a new method for feature selection that is based on a low-rank factorization of the transition matrix. Our approach derives features directly from high-dimensional raw inputs, such as image data. The method is easy to implement using SVD, and our experiments show that it is faster and more stable than alternative methods.

Corn cultivation location selection for bioethanol production: An application of BWM and extended PROMETHEE II

The purpose of this paper is to streamline the supply of corn, one of the key ingredients in the production of bioethanol. Using a comprehensive sustainable framework of criteria (including social, economic, and environmental dimensions), the potential locations of corn cultivation in Iran are examined from a multi-criteria decision analysis perspective. To this end, we need to find the importance of the relevant evaluation criteria and a ranking method to rank the potential locations. Best worst method (BWM) is used to determine the weight of the criteria presented in the framework based on the opinion of a sample of Iranian experts, after which the ‘preference ranking organization method for enrichment evaluations II’ (PROMETHEE II) is applied to ranking the different provinces of Iran. To improve the ranking results, we extend the PROMETHEE II by employing a set of piecewise linear value functions, for which the performance of alternatives with respect to the criteria calculated by the piecewise linear value functions is used to determine the amount of deviation in the first step of PROMETHEE II. The results of the hybrid methodology indicate that the presence of water, land cost and air pollution are the most important factors determining the ranking of the alternatives, and that Kordestan is the best province for corn cultivation in Iran.

Assessing the Bonitet of Cadastral Parcels for Land Reallocation in Urban Consolidation

Cadastral parcels valuation is a very important aspect when defining the priorities for the selection of newly formed parcels in the reallocation processes. Land reallocation involves a new redistribution of land parcels in the project implementation area. In doing so, it is necessary to take into account the preferences of private owners, the value of cadastral parcels, and their bonitet values, whose determination is the basic task of this research. The main goal is to propose a model to assess the bonitet of private cadastral parcels based on Expert System (ES) of fuzzy logic within the knowledge component, which would reduce uncertainty and increase the objectivity of the evaluation. Expert knowledge is included in the evaluation process by defining weighting coefficients for optimizing the rule base, and linear and nonlinear value functions for criteria standardizing. By applying the newly formed ES in the bonitet assessment, with the created base of expert knowledge, the processes of estimating attribute values have been improved, especially in the form of reducing uncertainty in the assessment of urban land parcels, as well as increasing objectivity by involving a group of experts in the model creation. The proposed model also provides equal access to all stakeholders in the process of urban renewal with different requirements and desires. The model also provides support in conducting the negotiation procedures and planning of land reallocation implementation. The proposed model was tested on the field of the construction of the Campus of University of Split.

Evaluation of the antitumor drugs therapeutic value using the method of multicriterial analysis of decision-making

Aim. Development of a tool for the evaluation of the therapeutic value of anticancer drugs based on multi-criteria analysis of decision making.Materials and methods. The study included 4 main stages. At the first stage, based on the results of the publications analysis and the EVIDEM system, a list of therapeutic value criteria was formed. At the second stage, for each criterion, a survey of experts was used to develop a scale and to build a value function using the curve-fitting method. At the third stage, the weighing was carried out using the discrete choice experiment method and the statistical package "support.CEs" in R Studio. At the fourth stage, based on the obtained values of the weight coefficients and value functions, a simple linear additive model was constructed.Results. The authors developed a tool that included 5 criteria for the value of anticancer drugs: efficacy, safety, disease severity, ease of administration, and therapeutic need. For each criterion, a linear value function was built, which allowed the authors to obtain the value of the severity of the criterion in the range from 0 to 1 based on the absolute values of the criterion for a particular drug. As a result of the weighing, two sets of weighting coefficients were obtained by using two different methods of survey results processing. The criterion of efficacy has the most significant contribution to the therapeutic value for the Russian health care system, and the least significant is the severity of the disease. The distribution of the criteria in the intermediate contribution ranks was significantly different between the two sets of weights.Conclusions. For practical application and implementation of the proposed tool, it is required to conduct the validation using the example of specific antitumor drugs to select a final set of weighting coefficients, check the stability of the results, and form a scale of therapeutic value levels.

Sparse Proximal Reinforcement Learning via Nested Optimization

We consider the tasks of feature selection and policy evaluation based on linear value function approximation in reinforcement learning problems. High-dimension feature vectors and limited number of samples can easily cause over-fitting and computation expensive. To prevent this problem, ${\ell _{1}}$ -regularized method obtains sparse solutions and thus improves generalization performance. We propose an efficient ${\ell _{1}}$ -regularized recursive least squares-based online algorithm with ${O}$ ( ${n} ^{{2}}$ ) complexity per time-step, termed ${\ell _{1}}$ -RC. With the help of nested optimization decomposition, ${\ell _{1}}$ -RC solves a series of standard optimization problems and avoids minimizing mean squares projected Bellman error with ${\ell _{1}}$ -regularization directly. In ${\ell _{1}}$ -RC, we propose RC with iterative refinement to minimize the operator error, and we propose an alternating direction method of multipliers with proximal operator to minimize the fixed-point error. The convergence of ${\ell _{1}}$ -RC is established based on ordinary differential equation method and some extensions are also given. In empirical computations, some state-of-the-art ${\ell _{1}}$ -regularized methods are chosen as the baselines, and ${\ell _{1}}$ -RC are tested in both policy evaluation and learning control benchmarks. The empirical results show the effectiveness and advantages of ${\ell _{1}}$ -RC.

An operational test for the existence of a consistent increasing quasi-concave value function

The existence of an increasing quasi-concave value function consistent with given preference information is an important issue in various fields including Economics, Multiple Criteria Decision Making, and Applied Mathematics. A quasi-concave value function is equivalent to convex-to-the-origin indifference contours, a very common assumption in Economics. In this paper, we establish necessary and sufficient conditions for the existence of a value function satisfying aforementioned properties. This leads to an operational, tractable and easy to use test for checking the existence of such a value function. In addition to developing the existence test, we construct consistent linear and non-linear desirable value functions.

Supply Inadequacy Risk Evaluation of Stand-Alone Renewable Powered Heat-Electricity Energy Systems: A Data-Driven Robust Approach

Integration of heat and electricity supply improves the overall energy efficiency and system operational flexibility. The renewable powered heat-electricity energy system is a promising way to set up residential energy supply facilities in remote areas beyond the reach of power system infrastructures. However, the volatility of wind and solar energy brings about the risk of supply inadequacy. This article proposes a data-driven robust method to quantify two measures of such a risk in the stand-alone renewable powered heat-electricity energy system. The uncertainty of renewable generation is modeled through a family of ambiguous probability distributions around an empirical one based on the Wasserstein metric; then, the probability of heat and electricity load shedding during a short period and related penalty cost are discussed. Through a polyhedral characterization of renewable power feasible region, the load shedding probability under the Wasserstein ambiguity set comes down to a linear program. With a piecewise linear optimal value function of the penalty cost, its expectation under the worst case distribution in the Wasserstein ambiguity set also gives rise to a linear program. The proposed method requires moderate information on renewable generation and makes full use of available data, whereas sustains computational tractability. The evaluation result is robust against the inaccuracy of renewable power distributions. Case studies demonstrate the effectiveness of the proposed approach.