Dynamic Value Function Research Articles

A Dynamic Principal-Agent Problem with One-Sided Commitment

In this paper, we consider a principal-agent problem where the agent is allowed to quit by incurring a cost. When the current agent quits the job, the principal will hire a new one, possibly with a different type. We characterize the principal’s dynamic value function, which could be discontinuous at the boundary, as the (unique) minimal solution of an infinite dimensional system of Hamilton-Jacobi-Bellman equations, parameterized by the agent’s type. This dynamic problem is time consistent in a certain sense. Some interesting findings are worth mentioning. First, self-enforcing contracts are typically suboptimal. The principal would rather let the agent quit and hire a new one. Next, the standard contract for a committed agent may also be suboptimal because of the presence of different types of agents in our model. The principal may prefer no commitment from the agent; then, the principal can hire a cheaper one from the market at a later time by designing the contract to induce the current agent to quit. Moreover, because of the cost incurred to the agent, the principal will see only finitely many quittings. Funding: J. Zhang is supported in part by the NSF [Grants DMS-1908665 and DMS-2205972]. Z. Zhu is supported by The Hong Kong University of Science and Technology (Guangzhou) Start-Up Fund [Grant G0101000240] and the Guangzhou-HKUST(GZ) Joint Funding Program [Grant 2024A03J0630].

Dynamic Labor Reallocation with Heterogeneous Skills and Uninsured Idiosyncratic Risk

Occupational specificity of human capital motivates an important role of occupational reallocation for the economy’s response to shocks and for the dynamics of inequality. We introduce occupational mobility, through a random choice model with dynamic value function optimization, into a multi-sector/multi-occupation Bewley (1980)- Aiyagari (1994) model with heterogeneous income risk, liquid and illiquid assets, price adjustment costs, and in which households differ by their occupation-specific skills. Labor income is a combination of endogenous occupational wages and idiosyncratic shock. Occupational reallocation and its impact on the economy depend on the transferability of workers’ skills across occupations and occupational specialization of the production function. The model matches well the statistics on income and wealth inequality, and the patterns of occupational mobility. It provides a laboratory for studying the short- and long-run effects of occupational shocks, automation and task encroaching on income and wealth inequality. We apply the model to the pandemic recession by adding an SIR block with occupation-specific infection risk and a ZLB policy and study the impact of occupational and aggregate labor supply shocks. We find that occupational mobility may tame the effect of the shocks but amplifies earnings inequality, as compared to a model without mobility.

Dynamic Labor Reallocation with Heterogeneous Skills and Uninsured Idiosyncratic Risk

Occupational specificity of human capital motivates an important role of occupational reallocation for the economy's response to shocks and for the dynamics of inequality. We introduce occupational mobility, through a random choice model with dynamic value function optimization, into a multi-sector/multi-occupation Bewley-Aiyagari model with heterogeneous income risk, liquid and illiquid assets, price adjustment costs, and in which households differ by their occupation-specific skills. Labor income is a combination of endogenous occupational wages and idiosyncratic shock. Occupational reallocation and its impact on the economy depend on the transferability of workers' skills across occupations and occupational specialization of the production function. The model matches well the statistics on income and wealth inequality, and the patterns of occupational mobility. It provides a laboratory for studying the short- and long-run effects of occupational shocks, automation and task encroaching on income and wealth inequality. We apply the model to the pandemic recession by adding an SIR block with occupation-specific infection risk and a ZLB policy and study the impact of occupational and aggregate labor supply shocks. We find that occupational mobility may tame the effect of the shocks but amplifies earnings inequality, as compared to a model without mobility.

Task scheduling algorithm based on value optimisation for anti‐missile phased array radar

A task scheduling algorithm based on value optimisation is proposed under comprehensive priority for anti-missile phased array radar. The model of threat degree is established for ballistic missile targets. The task comprehensive priority is obtained utilising a two-dimensional priority table which synthesises the target threat degree and the task deadline. Then, based on the comprehensive priority, each task gets a dynamic value function, instead of a fixed value as usual. In this way, the task value is enlarged when it is executed close to the desired execution time. A value optimisation model for task scheduling is constructed, and the genetic algorithm is utilised to solve this scheduling model. Thereby tasks can be executed as close as possible to the desired time, and the principle of scheduling timeliness can be better achieved. The performance of the proposed algorithm (PA) is compared with traditional scheduling algorithms by simulation experiments, as well as the impact that parameters of task value function have on scheduling performance is analysed. Simulation results show that compared to the traditional algorithms, the PA reduces the average time shift ratio and improves the value achieving ratio.

On the properties of dynamic value functions in the problem of optimal investment in incomplete markets

Abstract We study the analytical properties of a dynamic value function and of an optimal solution to the utility maximization problem in incomplete markets for utility functions defined on the whole real line. It was shown by Kramkov and Sirbu [Ann. Appl. Probab. 16 (2006), no. 3, 1352–1384] that if the relative risk-aversion coefficient of the utility function defined on the half real line is uniformly bounded away from zero and infinity, then the value function at time t = 0 of utility maximization problem is two-times differentiable and the optimal wealth is differentiable in probability with respect to the initial capital. Similar results are true for utility functions defined on the whole real line if instead of relative risk-aversion the same condition on the risk-aversion coefficient is imposed. Besides, assuming the continuity of the filtration F we prove that the derivative of the optimal wealth is strictly positive and that the derivative exists also in the sense of L 1-convergence. This enables us to show the existence of a strictly increasing (with respect to the initial capital) and absolutely continuous modification of the optimal wealth. We also study the differentiability properties of the value function V(t,x) and of the optimal wealth process Xt (x) for any t ∈ [0,T] and give the conditions under which the second derivative of the value function and the derivative of the optimal wealth process are continuous. We need these properties to show that the value function satisfies a certain backward stochastic partial differential equation used to characterize the optimal wealth process.

Humanoids Learning to Walk: A Natural CPG-Actor-Critic Architecture

The identification of learning mechanisms for locomotion has been the subject of much research for some time but many challenges remain. Dynamic systems theory (DST) offers a novel approach to humanoid learning through environmental interaction. Reinforcement learning (RL) has offered a promising method to adaptively link the dynamic system to the environment it interacts with via a reward-based value system. In this paper, we propose a model that integrates the above perspectives and applies it to the case of a humanoid (NAO) robot learning to walk the ability of which emerges from its value-based interaction with the environment. In the model, a simplified central pattern generator (CPG) architecture inspired by neuroscientific research and DST is integrated with an actor-critic approach to RL (cpg-actor-critic). In the cpg-actor-critic architecture, least-square-temporal-difference based learning converges to the optimal solution quickly by using natural gradient learning and balancing exploration and exploitation. Futhermore, rather than using a traditional (designer-specified) reward it uses a dynamic value function as a stability indicator that adapts to the environment. The results obtained are analyzed using a novel DST-based embodied cognition approach. Learning to walk, from this perspective, is a process of integrating levels of sensorimotor activity and value.

Capacity utilization under dynamic profit maximization

Empirically implementable measures of optimal capacity utilization are developed from the dynamic profit maximization perspective. The primal measure is based on behavior of the firm's supply along an optimal path to the steady state. The dual measure is based on the behavior of the dynamic value function in the stock of the quasi-fixed input. Rates of optimal capacity utilization are estimated for the U.S. food processing and distribution sector over the period 1948–1991.