Q-learning Reinforcement Learning Research Articles

Crop Yield Prediction Using Deep Reinforcement Learning Model for Sustainable Agrarian Applications

Predicting crop yield based on the environmental, soil, water and crop parameters has been a potential research topic. Deep-learning-based models are broadly used to extract significant crop features for prediction. Though these methods could resolve the yield prediction problem there exist the following inadequacies: Unable to create a direct non-linear or linear mapping between the raw data and crop yield values; and the performance of those models highly relies on the quality of the extracted features. Deep reinforcement learning provides direction and motivation for the aforementioned shortcomings. Combining the intelligence of reinforcement learning and deep learning, deep reinforcement learning builds a complete crop yield prediction framework that can map the raw data to the crop prediction values. The proposed work constructs a Deep Recurrent Q-Network model which is a Recurrent Neural Network deep learning algorithm over the Q-Learning reinforcement learning algorithm to forecast the crop yield. The sequentially stacked layers of Recurrent Neural network is fed by the data parameters. The Q- learning network constructs a crop yield prediction environment based on the input parameters. A linear layer maps the Recurrent Neural Network output values to the Q-values. The reinforcement learning agent incorporates a combination of parametric features with the threshold that assist in predicting crop yield. Finally, the agent receives an aggregate score for the actions performed by minimizing the error and maximizing the forecast accuracy. The proposed model efficiently predicts the crop yield outperforming existing models by preserving the original data distribution with an accuracy of 93.7%.

Comparison of conventional and rapid-acting antidepressants in a rodent probabilistic reversal learning task.

Deficits in reward processing are a central feature of major depressive disorder with patients exhibiting decreased reward learning and altered feedback sensitivity in probabilistic reversal learning tasks. Methods to quantify probabilistic learning in both rodents and humans have been developed, providing translational paradigms for depression research. We have utilised a probabilistic reversal learning task to investigate potential differences between conventional and rapid-acting antidepressants on reward learning and feedback sensitivity. We trained 12 rats in a touchscreen probabilistic reversal learning task before investigating the effect of acute administration of citalopram, venlafaxine, reboxetine, ketamine or scopolamine. Data were also analysed using a Q-learning reinforcement learning model to understand the effects of antidepressant treatment on underlying reward processing parameters. Citalopram administration decreased trials taken to learn the first rule and increased win-stay probability. Reboxetine decreased win-stay behaviour while also decreasing the number of rule changes animals performed in a session. Venlafaxine had no effect. Ketamine and scopolamine both decreased win-stay probability, number of rule changes performed and motivation in the task. Insights from the reinforcement learning model suggested that reboxetine led animals to choose a less optimal strategy, while ketamine decreased the model-free learning rate. These results suggest that reward learning and feedback sensitivity are not differentially modulated by conventional and rapid-acting antidepressant treatment in the probabilistic reversal learning task.

Wind and Storage Cooperative Scheduling Strategy Based on Deep Reinforcement Learning Algorithm

Wind energy has become one of the most promising new energy sources in the context of the global energy interconnection. However, the inherent uncertainty of wind power and the volatility of its output make it difficult to join the internet for scheduling. The uncertainty of the output must be reduced or even eliminated by certain methods. In this paper, wind power and energy storage are coordinated to eliminate the uncertainty of wind power output, and to reduce the burden on the grid while ensuring the long-term operation of wind farms. This paper first introduces Q-learning in reinforcement learning as a controller. Through a large number of historical wind power data training, the controller has good decision-making ability, so as to reduce the punishment caused by wind power uncertainty; then the Q-learning is improved aiming at the maximum average income of the stage. Finally, the Q-value network is established with conventional Q-learning and improved Q-learning. The DQN algorithm in the deep reinforcement learning algorithm is introduced for deep training and decision-making and the three algorithms are verified. The result proves that the deep reinforcement learning algorithm can achieve better control effect than Q-learning.

An intelligent wireless channel allocation in HAPS 5G communication system based on reinforcement learning

Channel allocation is the prerequisite for the HAPS (high-altitude platform station) 5G communication network to transmit information. An intelligent wireless channel allocation algorithm for HAPS 5G massive MIMO (multiple-input multiple-output) system based on reinforcement learning was proposed. Q-learning reinforcement learning algorithm and the back-propagation neural network were combined, which enabled HAPS 5G massive MIMO systems autonomous learn according to the environment and allocate channel resources of the system efficiently. Agents perceive the state information in the channel environment through continuous interaction with the channel environment, learn from the environment state to the action mapping, and use the back-propagation neural network for learning training, use the neural network instead of the Q value table, and train the network with each Q update as a training example to update the evaluation function. It effectively improves the overall performance of the system.

Context aware Q-Learning-based model for decision support in the negotiation of energy contracts

Automated negotiation plays a crucial role in the decision support for bilateral energy transactions. In fact, an adequate analysis of past actions of opposing negotiators can improve the decision-making process of market players, allowing them to choose the most appropriate parties to negotiate with in order to increase their outcomes. This paper proposes a new model to estimate the expected prices that can be achieved in bilateral contracts under a specific context, enabling adequate risk management in the negotiation process. The proposed approach is based on an adaptation of the Q-Learning reinforcement learning algorithm to choose the best scenario (set of forecast contract prices) from a set of possible scenarios that are determined using several forecasting and estimation methods. The learning process assesses the probability of occurrence of each scenario, by comparing each expected scenario with the real scenario. The final chosen scenario is the one that presents the higher expected utility value. Besides, the learning method can determine which is the best scenario for each context, since the behaviour of players can change according to the negotiation environment. Consequently, these conditions influence the final contract price of negotiations. This approach allows the supported player to be prepared for the negotiation scenario that is the most probable to represent a reliable approximation of the actual negotiation environment.

Development of a Smart Home Context-aware Application: A Machine Learning based Approach

Context-awareness is an important characteristic of smart home. Several methods are used in context-aware application to provide services. The main target of smart home is to predict the demand of home users and proactively provide the proper services by computing user’s context information. In this paper, we present a context-aware application which can provide service according to predefined choice of user. It uses Mahalanobis distance based k nearest neighbors classifier technique for inference of predefined service. We combine the features of supervised and unsupervised machine learning in the proposed application. This application can also adapt itself when the choice of user is changed by using Q-learning reinforcement learning algorithm.

FUZZY STATE AGGREGATION AND POLICY HILL CLIMBING FOR STOCHASTIC ENVIRONMENTS

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing.