Model Of Optimal Policy Research Articles

The Deep Reinforcement Learning-Based VR Training System With Haptic Guidance for Catheterization Skill Transfer

With growing demands in endovascular intervention, priority is expected to be given to the catheter manipulation skill training. For novices, it is inevitable to face the dilemma that progress is achieved slowly when they practice repeatedly. Thus, exploring underlying behavior patterns from experts’ demonstrations and transferring manipulation skills to novice are of significance when designing catheterization training systems. To address such issues, a deep reinforcement learning-based VR training system has been presented, which aims to exploit expert’s experience-related skills to correct novices’ actions via the 2-DOF haptic guidance. The training system mainly consists of three parts: a force sensors’ integrated catheter manipulator, a VR simulator, and a learning-based model. As experience-related skills are encoded as the correlation between operator–tool and tool–tissue interactions, the deep deterministic policy gradient (DDPG), one of deep reinforcement learning algorithms, is adopted so that expert’s skills can be extracted as the optimal policy model by learning from the catheterization demonstrations. Based on such learning model, the catheter manipulator can provide the intuitive haptic guidance to online correct novices’ catheterization maneuvers both in translational and rotational directions. To evaluate the skill improvement, a five-day training session has been carried out and the results reveal that prominent progress has been made with respect to safety operation and task proficiency. Moreover, substantial evidences support the finding that the newly developed training system is qualified to transfer experience-related skills from expert to novice because trainees are capable of making wise decisions independently without any haptic cues after training sessions.

Recursive Contracts

We obtain a recursive formulation for a general class of optimization problems with forward‐looking constraints which often arise in economic dynamic models, for example, in contracting problems with incentive constraints or in models of optimal policy. In this case, the solution does not satisfy the Bellman equation. Our approach consists of studying a recursive Lagrangian. Under standard general conditions, there is a recursive saddle‐point functional equation (analogous to a Bellman equation) that characterizes a recursive solution to the planner's problem. The recursive formulation is obtained after adding a co‐state variable μ t summarizing previous commitments reflected in past Lagrange multipliers. The continuation problem is obtained with μ t playing the role of weights in the objective function. Our approach is applicable to characterizing and computing solutions to a large class of dynamic contracting problems.

Environmental Quality and Economic Development

Economic growth invariably leads to environmental degradation. But in this paper it is shown that economic development may be necessary before a country will adopt a policy of environmental protection. Within the context of an optimal-policy model, this proposition is proved and an optimal policy is derived. It is shown that knowledge of preferences for environmental quality is necessary for such a policy of environmental protection. It is then shown how such preferences can be estimated from market demand. The estimation is carried out with select consumption data.

The balance of payments and optimal economic policy: The assignment problem revisited

In this paper a policy model suggested by Niehans is extended to show the particular and non-optimal character of the so-called assignment rules in the theory of economic policy in open economies. From the assumption that the government has two arguments in its objective function — income and the exchange reserves — an optimal policy model is developed with the techniques of optimal control theory. The optimal control rules derived from this model are easily compared with the traditional assignment rules. The model also shows how rational behaviour of the policymakers could lead to the creation of a political business cycle.