Nonlinear Value Function Research Articles

Low-carbon economic analysis of a virtual power plant with wind and solar power considering the integrated flexible operation mode of a carbon capture thermoelectric unit

During the period when heating is required in North China, the task of meeting the heating needs of customers is mainly undertaken by thermoelectric units under the ”determining electricity by heat” operation mode. This heating mode leads to difficulties in incorporating renewable energy and high carbon dioxide emissions. To solve these problems, carbon capture technology and wind power heating technology are introduced in this study. Cogeneration units with a carbon capture integrated flexible operation mode are introduced into the system to reduce CO2 emissions, while making the electric output and the thermal output more flexible. Then, the cogeneration optimal scheduling model of a virtual power plant is constructed. This model includes wind power heating equipment and carbon capture cogeneration units under the integrated flexible operation mode. Due to the highly nonlinear characteristics of the model, we solve this problem by the Deep Q-Network (DQN) algorithm, which is an algorithm that uses the nonlinear approximation value function of a neural network to make the model more robust. Finally, we propose three technical strategies and compare the results of the three strategies in terms of low carbon levels and economic value. The results show that the model can effectively improve the economic income of the system and reduce carbon emissions.

Online and hard constrained adaptive dynamic programming algorithm for energy storage control in smart buildings

AbstractThe high computational complexity caused by static optimization is the key factor to hinder the development of energy management systems. Adaptive dynamic programming (ADP) is an effective dynamic optimization method to break through the bottleneck. However, the hard constraints of energy system have not been fully considered due to the non‐convexity and nonlinearity of the value function, which makes the theoretical analysis complicated and brings safe security problems in battery systems. In this article, a systematic online ADP control framework is proposed for smart buildings control to ensure hard constraints to be satisfied. The second‐order local expansion at the current state is used to replace the nonlinear value function to simplify the theoretical analysis with the error of the reminder term. Based on the local property of value function, a method for the determination of adaptive parameters is first designed. It is proven that the solution of adaptive parameters not only prevents over‐charged and over‐discharged of the battery but also limits the charging and discharging power of the battery to be less than the rated power. In addition, long‐short term memory (LSTM) neural networks, a type dynamic network with memory characteristics, are used for the implementation of the present algorithm instead of the static networks to help realize the algorithm online. Due to the hidden state of LSTM, the performance of the online algorithm is improved after running the energy system. Numerical results verify the effectiveness of the proposed online control algorithm.

Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases.

A theoretical framework for the reinforcement learning of aesthetic biases was recently proposed based on brain circuitries revealed by neuroimaging. A model grounded on that framework accounted for interesting features of human aesthetic biases. These features included individuality, cultural predispositions, stochastic dynamics of learning and aesthetic biases, and the peak-shift effect. However, despite the success in explaining these features, a potential weakness was the linearity of the value function used to predict reward. This linearity meant that the learning process employed a value function that assumed a linear relationship between reward and sensory stimuli. Linearity is common in reinforcement learning in neuroscience. However, linearity can be problematic because neural mechanisms and the dependence of reward on sensory stimuli were typically nonlinear. Here, we analyze the learning performance with models including optimal nonlinear value functions. We also compare updating the free parameters of the value functions with the delta rule, which neuroscience models use frequently, vs. updating with a new Phi rule that considers the structure of the nonlinearities. Our computer simulations showed that optimal nonlinear value functions resulted in improvements of learning errors when the reward models were nonlinear. Similarly, the new Phi rule led to improvements in these errors. These improvements were accompanied by the straightening of the trajectories of the vector of free parameters in its phase space. This straightening meant that the process became more efficient in learning the prediction of reward. Surprisingly, however, this improved efficiency had a complex relationship with the rate of learning. Finally, the stochasticity arising from the probabilistic sampling of sensory stimuli, rewards, and motivations helped the learning process narrow the range of free parameters to nearly optimal outcomes. Therefore, we suggest that value functions and update rules optimized for social and ecological constraints are ideal for learning aesthetic biases.

Basic Geometric Dispersion Theory of Decision Making Under Risk: Asymmetric Risk Relativity, New Predictions of Empirical Behaviors, and Risk Triad

Many studies have spotlighted significant applications of expected utility theory (EUT), cumulative prospect theory (CPT), and mean-variance in assessing risks. We illustrate that these models and their extensions are unable to predict risk behaviors accurately in out-of-sample empirical studies. EUT uses a nonlinear value (utility) function of consequences but is linear in probabilities, which has been criticized as its primary weakness. Although mean-variance is nonlinear in probabilities, it is symmetric, contradicts first-order stochastic dominance, and uses the same standard deviation for both risk aversion and risk proneness. In this paper, we explore a special case of geometric dispersion theory (GDT) that is simultaneously nonlinear in both consequences and probabilities. It complies with first-order stochastic dominance and is asymmetric to represent the mixed risk-averse and risk-prone behaviors of the decision makers. GDT is a triad model that uses expected value, risk-averse dispersion, and risk-prone dispersion. GDT uses only two parameters, z and zX; these constants remain the same regardless of the scale of risk problem. We compare GDT to several other risk dispersion models that are based on EUT and/or mean-variance, and identify verified risk paradoxes that contradict EUT, CPT, and mean-variance but are easily explainable by GDT. We demonstrate that GDT predicts out-of-sample empirical risk behaviors far more accurately than EUT, CPT, mean-variance, and other risk dispersion models. We also discuss the underlying assumptions, meanings, and perspectives of GDT and how it reflects risk relativity and risk triad. This paper covers basic GDT, which is a special case of general GDT of Malakooti [Malakooti (2020) Geometric dispersion theory of decision making under risk: Generalizing EUT, RDEU, & CPT with out-of-sample empirical studies. Working paper, Case Western Reserve University, Cleveland.].

Assessing the Bonitet of Cadastral Parcels for Land Reallocation in Urban Consolidation

Cadastral parcels valuation is a very important aspect when defining the priorities for the selection of newly formed parcels in the reallocation processes. Land reallocation involves a new redistribution of land parcels in the project implementation area. In doing so, it is necessary to take into account the preferences of private owners, the value of cadastral parcels, and their bonitet values, whose determination is the basic task of this research. The main goal is to propose a model to assess the bonitet of private cadastral parcels based on Expert System (ES) of fuzzy logic within the knowledge component, which would reduce uncertainty and increase the objectivity of the evaluation. Expert knowledge is included in the evaluation process by defining weighting coefficients for optimizing the rule base, and linear and nonlinear value functions for criteria standardizing. By applying the newly formed ES in the bonitet assessment, with the created base of expert knowledge, the processes of estimating attribute values have been improved, especially in the form of reducing uncertainty in the assessment of urban land parcels, as well as increasing objectivity by involving a group of experts in the model creation. The proposed model also provides equal access to all stakeholders in the process of urban renewal with different requirements and desires. The model also provides support in conducting the negotiation procedures and planning of land reallocation implementation. The proposed model was tested on the field of the construction of the Campus of University of Split.

Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments

In this paper, dynamic non-cooperative coexistence between a cognitive pulsed radar and a nearby communications system is addressed by applying nonlinear value function approximation via deep reinforcement learning (Deep RL) to develop a policy for optimal radar performance. The radar learns to vary the bandwidth and center frequency of its linear frequency modulated (LFM) waveforms to mitigate mutual interference with other systems and improve target detection performance while also maintaining sufficient utilization of the available frequency bands required for a fine range resolution. We demonstrate that our approach, based on the Deep Q-Learning (DQL) algorithm, enhances important radar metrics, including SINR and bandwidth utilization, more effectively than policy iteration or sense-and-avoid (SAA) approaches in a variety of realistic coexistence environments. We also extend the DQL-based approach to incorporate Double Q-learning and a recurrent neural network to form a Double Deep Recurrent Q-Network (DDRQN). We demonstrate the DDRQN results in favorable performance and stability compared to DQL and policy iteration. Finally, we demonstrate the practicality of our proposed approach through a discussion of experiments performed on a software defined radar (SDRadar) prototype system. Our experimental results indicate that the proposed Deep RL approach significantly improves radar detection performance in congested spectral environments when compared to policy iteration and SAA.

An operational test for the existence of a consistent increasing quasi-concave value function

The existence of an increasing quasi-concave value function consistent with given preference information is an important issue in various fields including Economics, Multiple Criteria Decision Making, and Applied Mathematics. A quasi-concave value function is equivalent to convex-to-the-origin indifference contours, a very common assumption in Economics. In this paper, we establish necessary and sufficient conditions for the existence of a value function satisfying aforementioned properties. This leads to an operational, tractable and easy to use test for checking the existence of such a value function. In addition to developing the existence test, we construct consistent linear and non-linear desirable value functions.

Dynamic Assortment Optimization for Reusable Products with Random Usage Durations

We consider dynamic assortment problems with reusable products, in which each arriving customer chooses a product within an offered assortment, uses the product for a random duration of time, and returns the product back to the firm to be used by other customers. The goal is to find a policy for deciding on the assortment to offer to each customer so that the total expected revenue over a finite selling horizon is maximized. The dynamic-programming formulation of this problem requires a high-dimensional state variable that keeps track of the on-hand product inventories, as well as the products that are currently in use. We present a tractable approach to compute a policy that is guaranteed to obtain at least 50% of the optimal total expected revenue. This policy is based on constructing linear approximations to the optimal value functions. When the usage duration is infinite or follows a negative binomial distribution, we also show how to efficiently perform rollout on a simple static policy. Performing rollout corresponds to using separable and nonlinear value function approximations. The resulting policy is also guaranteed to obtain at least 50% of the optimal total expected revenue. The special case of our model with infinite usage durations captures the case where the customers purchase the products outright without returning them at all. Under infinite usage durations, we give a variant of our rollout approach whose total expected revenue differs from the optimal by a factor that approaches 1 with rate cubic-root of Cmin, where Cmin is the smallest inventory of a product. We provide computational experiments based on simulated data for dynamic assortment management, as well as real parking transaction data for the city of Seattle. Our computational experiments demonstrate that the practical performance of our policies is substantially better than their performance guarantees and that performing rollout yields noticeable improvements.This paper was accepted by Yinyu Ye, optimization.

Prospect Theory and the Newsvendor Problem with Mental Accounting

The manner in which economic outcomes are coded in a value function is critical because it has substantial influence on the evaluation of risky choices. In this paper, we formulate multiple mental accounts in a nonlinear value function and propose a newsvendor problem with mental accounting to predict and explain pull-to-center phenomenon. We show that the distinctive predictions of the proposed model come from the value function with mental accounting. We identify the individual and combined effects of loss aversion, risk aversion, and risk seeking on shaping newsvendor ordering behavior. Our work demonstrates that prospect theory can explain the decision bias and ordering behavior observed in newsvendor experiments. We also provide some additional insights to explain the studies in the literature.

A green approach for the removal of Sr(II) from aqueous media: Kinetics, isotherms and thermodynamic studies

Adsorption efficiency of native, NaOH-treated and immobilized peanut husk and sugarcane bagasse for Sr(II) removal was studied in batch mode. In view of promising adsorption efficiency of peanut husk versus sugarcane bagasse, the biosorption behaviour of the Sr(II) ions onto peanut husk (native and modified) was studied as a function of pH, biosorbent dose, contact time, initial metal ion concentration and temperature for the maximum removal of Sr(II) ions. Linear and non-linear pseudo-first and second-order kinetic models were applied and value of R2 and six non-linear regression error functions, namely hybrid fractional error function (HYBRID), Marquardt's percent standard deviation (MPSD), average relative error (ARE), the sum of the errors squared (ERRSQ/SSE), the sum of the absolute errors (EABS) and Chi-square test (χ2) were used to predict the most optimum kinetic model. Sorbent-sorbate reaction nature was estimated by fitting equilibrium data by non-linear and transformed linear forms of the Langmuir, Freundlich and Redlich-Peterson isotherms and most optimum isothermal model was optimized by comparing linear and non-linear R2 value and non-linear regression error functions. HCl proved most successful eluating agents for sorbed Sr(II) ions. Biosorption characteristics and effectiveness of the process was also confirmed by Fourier transform infra-red spectroscopy (FTIR), scanning electron microscopy and energy dispersive X-ray spectroscopy (SEM-EDX). In view of promising efficiency of peanut husk as an adsorbent, it could possibly be used for the removal of Sr(II) ions from aqueous medium and is also extendable to other radionuclide.

Sustainability Performance Measurement with Data Envelopment Analysis: A Combined Production- and Decision-Theoretic Approach

Decision-based production theory is a generalization of traditional production theories developed in order to integrate concerns of modern management science and economics, such as sustainability and environmental protection. The main idea is the distinction of technologically determined inputs and outputs of a production system’s activity from its desired or undesired impacts on (artificial or natural) environments, formalized by multiple value functions. The paper extends the literature on the generalization and theoretical foundation of data envelopment analysis (DEA) by decision-based production theory. It provides fruitful insights into some well-known pitfalls and protocols in DEA and is practically useful for the measurement of effectiveness in achieving different, often conflicting sustainability objectives. New results are concerned with hierarchies of DEA evaluations, undesired input and output objects, superfluous or inadequate disposability assumptions as well as problems of nonlinear value functions.

Four Common Simplifications of Multi-Criteria Decision Analysis do not hold for River Rehabilitation.

River rehabilitation aims at alleviating negative effects of human impacts such as loss of biodiversity and reduction of ecosystem services. Such interventions entail difficult trade-offs between different ecological and often socio-economic objectives. Multi-Criteria Decision Analysis (MCDA) is a very suitable approach that helps assessing the current ecological state and prioritizing river rehabilitation measures in a standardized way, based on stakeholder or expert preferences. Applications of MCDA in river rehabilitation projects are often simplified, i.e. using a limited number of objectives and indicators, assuming linear value functions, aggregating individual indicator assessments additively, and/or assuming risk neutrality of experts. Here, we demonstrate an implementation of MCDA expert preference assessments to river rehabilitation and provide ample material for other applications. To test whether the above simplifications reflect common expert opinion, we carried out very detailed interviews with five river ecologists and a hydraulic engineer. We defined essential objectives and measurable quality indicators (attributes), elicited the experts´ preferences for objectives on a standardized scale (value functions) and their risk attitude, and identified suitable aggregation methods. The experts recommended an extensive objectives hierarchy including between 54 and 93 essential objectives and between 37 to 61 essential attributes. For 81% of these, they defined non-linear value functions and in 76% recommended multiplicative aggregation. The experts were risk averse or risk prone (but never risk neutral), depending on the current ecological state of the river, and the experts´ personal importance of objectives. We conclude that the four commonly applied simplifications clearly do not reflect the opinion of river rehabilitation experts. The optimal level of model complexity, however, remains highly case-study specific depending on data and resource availability, the context, and the complexity of the decision problem.

Re-use of agricultural wastes for the removal and recovery of Zr(IV) from aqueous solutions

This study assesses the feasibility of Zr(IV) removal and recovery from aqueous solutions by novel biosorbents prepared from selected agricultural wastes. Sugarcane bagasse was selected for further investigation after showing increased biosorption capacity during the initial screening experiment. The biosorption efficiency of native (untreated), SDS-treated and immobilised bagasse for Zr(IV) removal was studied and optimization of the experimental conditions carried out including pH, biosorbent weight, contact time, initial metal ion concentration and temperature to maximise adsorption. Sorbent–sorbate reaction behaviour was estimated by fitting equilibrium data by non-linear and transformed linear forms of the Langmuir, Freundlich and Redlich–Peterson isotherms as well as pseudo-first and second-order kinetic models. The best fitting isothermal or kinetic model was optimized by comparing linear and non-linear R2 value and non-linear regression error functions. H2SO4 proved to be the most effective desorbing agent in recovery of the sorbed Zr(IV) ions from all forms of bagasse. Biosorbent characterisation and effectiveness of the process was confirmed by Fourier transform infra-red spectroscopy (FT-IR), scanning electron microscopy and energy dispersive X-ray spectroscopy (SEM-EDX). The data illustrate that native (untreated), SDS-treated and immobilised bagasse have great potential to remove and recover Zr from wastewater.

Double Helix Value Functions, Ordinal/Cardinal Approach, Additive Utility Functions, Multiple Criteria, Decision Paradigm, Process, and Types (Z Theory I)

Z Utility Theory refers to a class of nonlinear utility functions for solving Risk and Multiple Criteria Decision-Making problems. Z utility functions are hybrids of additive and nonadditive (nonlinear) functions. This paper addresses the concepts and assessment methods for the additive part of Z-utility functions for multiple criteria problems that satisfy the efficiency (nondominancy) principle. We provide a decision paradigm and guidelines on how to approach, formulate, and solve decision-making problems. We, also, overview the modeling of decision process based on four types of decision-making styles. For multi-criteria problems, a new definition of convex efficiency is introduced. Also polyhedral efficiency is developed for presenting multi-criteria efficiency (nondominancy) graphically. New double helix quasi-linear value functions for multi-criteria are developed. Two types of double helix value functions for solving bi-criteria (Advantages versus Disadvantages) and also risk problems are introduced: Food–Fun curves for expected values and Fight-Flight curves for expected risk values. Ordinal/Cardinal Approach (OCA) for assessment of additive utility functions is developed. Simple consistency tests to determine whether the assessed utility function satisfies ordinal and/or cardinal properties are provided. We show that OCA can also be used to solve outranking problems. We provide a critique of Analytic Hierarchy Process (AHP) for assessing additive value functions and show that the developed Ordinal/Cardinal Approach overcomes the shortcomings of AHP. We also develop a unified/integrated approach for simultaneous assessment of nonlinear value and additive (multi-criteria) utility functions. These results in an additive utility function that can be concave, convex, or hybrid concave/convex based on the nonlinear value function. Finally, we show an interactive paired comparisons approach for solving nonadditive and nonlinear utility functions for bi-criteria decision-making problems. Several illustrative examples are provided. The paper provides reliable and robust approaches for modeling the utility preferences of heterogeneous economic agents in macro and micro-economics.

Simulation-Based Approximate Policy Iteration with Generalized Logistic Functions

We present an approximate dynamic programming method based on simulation, policy iteration, a postdecision state formulation, and a logistic value function approximation. This method was developed as part of our efforts to determine whether nonlinear value function approximations could provide cost-effective policies for advance patient scheduling problems, and as a way of identifying the main advantages and disadvantages of using simulation versus linear programming to approximately solve dynamic capacity allocation problems. We first apply the proposed method to a queueing problem and then study a more practical application based on an advance multipriority patient scheduling problem. We investigate the quality and practical implications of the resulting appointment scheduling policies using simulation, and compare their performance to that of four other policies. Patient scheduling policies obtained by the new method not only depend on the number of appointments already booked on each day but also on the overall system workload. In particular, these policies provide lower discounted cost values and shorter average wait times for higher priority patients than policies directly obtained using linear programming and an affine value function approximation in the predecision state variables.

A Spatial Multi-Criteria Model for the Evaluation of Land Redistribution Plans

A planning support system for land consolidation has been developed that has, at its heart, an expert system called LandSpaCES (Land Spatial Consolidation Expert System) which contains a “design module” that generates alternative land redistributions under different scenarios and an “evaluation module” which integrates GIS with multi-criteria decision making for assessing these alternatives. This paper introduces the structural framework of the latter module which has been applied using a case study in Cyprus. Two new indices are introduced: the “parcel concentration coefficient” for measuring the dispersion of parcels; and the “landowner satisfaction rate” for predicting the acceptance of the land redistribution plan by the landowners in terms of the location of their new parcels. These two indices are used as criteria for the evaluation of the land redistribution alternatives and are transferable to any land consolidation project. Moreover, a modified version of the ratio estimation procedure, referred to as the “qualitative rating method” for assigning weights to the evaluation criteria, is presented, along with a set of non-linear value functions for standardizing the performance scores of the alternatives and incorporating expert knowledge for five evaluation criteria. The application of the module showed that it is a powerful new tool for the evaluation of alternative land redistribution plans that could be implemented in other countries after appropriate adjustments. A broader contribution has also been made to spatial planning processes, which might follow the methodology and innovations presented in this paper.

Accounting for Nonlinearity in the MCDM Approach for a Transportation Planning Application

We assess the impact of accounting for nonlinearity of selected value functions to improve the multiattribute utility theory approach for transportation planning applications. Research findings were obtained by conducting a case study for investigating and improving the project evaluation and selection with the collaboration of a state-level transportation agency. A methodology based on the analytic hierarchy process has been used for elicitation of the nonlinear value functions. It was found that employing nonlinear functions, as opposed to the commonly assumed linear scaling, even for only a subset of performance measures, had a significant impact in calculating the projects' index final scores, with changes ranging from 3% to 26%, possibly overturning the project selection. This paper highlights the importance that needs to be given to construct the value functions in decision-making problems and shows how the process could be improved by employing available tools properly.

Data envelopment analysis with nonlinear virtual inputs and outputs

An underlying assumption in DEA is that the weights coupled with the ratio scales of the inputs and outputs imply linear value functions. In this paper, we present a general modeling approach to deal with outputs and/or inputs that are characterized by nonlinear value functions. To this end, we represent the nonlinear virtual outputs and/or inputs in a piece-wise linear fashion. We give the CCR model that can assess the efficiency of the units in the presence of nonlinear virtual inputs and outputs. Further, we extend the models with the assurance region approach to deal with concave output and convex input value functions. Actually, our formulations indicate a transformation of the original data set to an augmented data set where standard DEA models can then be applied, remaining thus in the grounds of the standard DEA methodology. To underline the usefulness of such a new development, we revisit a previous work of one of the authors dealing with the assessment of the human development index on the light of DEA.

Explaining outcome type interactions with frame: Aspiration level and the value function

Research on framing effects has revealed cases where the type of outcome at risk (e.g., human lives vs. animal lives) affects the magnitude of the framing effect. Some authors have appealed to the shape of the value function as predicting when framing effects will occur: The more valuable the outcome type, the more nonlinear its value function, and the larger the resulting framing effect (Levin & Chapman, 1990). However, having a more or less nonlinear value function cannot explain situations in which participants strongly prefer the same option in both frames. Another factor that may be at work in these types of outcome effects is an aspiration level (AL; Lopes, 1987; Schneider, 1992), which determines how acceptable the options are and combines (or competes) with the risk attitude encouraged by frame. The results described here indicate that differences in the shape of the value function between outcome types are evident but are inconsistent between framed losses and gains, though nonlinearity in the value function can be increased with a manipulation that also encourages framing effects. The results also demonstrate that an AL can lead to the same predominant risk preference in the positive and negative frame. These findings indicate that the shape of the value function and the AL each play a role in outcome type interactions with frame, and in some cases, a combination of the two factors may be at work.

Implementation of the Newsvendor Model with Clearance Pricing: How to (and How Not to) Estimate a Salvage Value

The newsvendor model is designed to decide how much of a product to order when the product is to be sold over a short selling season with stochastic demand and there are no additional opportunities to replenish inventory. There are many practical situations that reasonably conform to those assumptions, but the traditional newsvendor model also assumes a fixed salvage value: all inventory left over at the end of the season is sold off at a fixed per-unit price. The fixed salvage value assumption is questionable when a clearance price is rationally chosen in response to the events observed during the selling season: a deep discount should be taken if there is plenty of inventory remaining at the end of the season, whereas a shallow discount is appropriate for a product with higher than expected demand. This paper solves for the optimal order quantity in the newsvendor model, assuming rational clearance pricing. We then study the performance of the traditional newsvendor model. The key to effective implementation of the traditional newsvendor model is choosing an appropriate fixed salvage value. (We show that an optimal order quantity cannot be generally achieved by merely enhancing the traditional newsvendor model to include a nonlinear salvage value function.) We demonstrate that several intuitive methods for estimating the salvage value can lead to an excessively large order quantity and a substantial profit loss. Even though the traditional model can result in poor performance, the model seems as if it is working correctly: the order quantity chosen is optimal given the salvage value inputted to the model, and the observed salvage value given the chosen order quantity equals the inputted one. We discuss how to estimate a salvage value that leads the traditional newsvendor model to the optimal or near-optimal order quantity. Our results highlight the importance of understanding how a model can interact with its own inputs: when inputs to a model are influenced by the decisions of the model, care is needed to appreciate how that interaction influences the decisions recommended by the model and how the model’s inputs should be estimated.