Constant Box Research Articles

Q-Learning with the Variable Box Method: A Case Study to Land a Solid Rocket

Some critical tasks require refined actions near the target, for instance, steering a car in a crowded parking lot or landing a rocket. These tasks are critical because failure to comply with the constraints near the target may lead to a fatal (unrecoverable) condition. Thus, a higher resolution action is required near the target to increase maneuvering precision. Moreover, completing the task becomes more challenging if the environment changes or is uncertain. Therefore, novel approaches have been proposed for these problems. In particular, reinforcement learning schemes such as Q-learning have been suggested to learn from scratch, subject to exploring action–state causal relationships aimed at action decisions that lead to an increase in the reward. Q-learning refines iterative action inputs by exploring state spaces that maximize the reward. However, reducing the (constant) resolution box needed for critical tasks increases the computational load, which may lead to the tantamount curse of the dimensionality problem. This paper proposes a variable box method to maintain a low number of boxes but reduce its resolution only near the target to increase action resolution as needed. The proposal is applied to a critical task such as landing a solid rocket, whose dynamics are highly nonlinear, underactuated, non-affine, and subject to environmental disturbances. Simulations show successful landing without leading to a curse of dimensionality, typical of the classical (constant box) Q-learning scheme.

고온환경 작업을 위한 펠티어 소자 냉각복 개발 및 쾌적성 평가

This paper reports on a prototype cooling garment applying a cooling module. The cooling module was composed of a Peltier device, a cold sink, a heat sink and two fans. A constant box was used to evaluate the cooling effect of the module. Two cooling modules were attached on each side of the garment. The wear trial was conducted using 10 male subjects in an environmental chamber maintained at , RH. Subjective sensations of thermal, humidity, and comfort were surveyed. Statistical package SPSS12.0 was used for the t-test and the Wilcoxon signed-rank test. The results showed that most effective cooling module decreased the temperature of the constant temperature box by . The micro-temperature of the cooling garment with a Peltier device was lower than the control garment during the exercise. In particular, the chest skin temperature was lower with the cooling garment than the control. The maximum temperature difference was on the sides of the layer. Subjective thermal sensation from wear trials of the Peltier device attached garment was lower than the control garment. Subjects felt more comfortable with the cooling garment in almost all the periods.