Reward Approach Research Articles

Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards.

An Effort-Based Reward Approach for Allocating Load Shedding Amount in Networked Microgrids Using Multiagent System

An effort-based reward approach is proposed in this article for the allocation of load shedding amount in interconnected microgrids. In this article, effort is defined as the relative contribution of a microgrid to the network with respect to its capacity. In contrast to the absolute contribution-based reward methods, where smaller microgrids are discriminated, the proposed effort-based reward method provides a fair chance for microgrids of all sizes. Historic data of efforts of each microgrid are recorded and an index (effort index) is formulated. The effort index is used as a measure to allocate load shedding to microgrids during emergencies, where microgrids with higher effort indices receive lesser load shedding and vice versa. Different weights are defined for efforts of microgrids depending on the operation mode of the network due to the difference in the importance of power sharing in each mode. In addition, a reward compensation algorithm is devised to mitigate gaining of benefits for longer times while making lesser contributions. The proposed method is realized by using a multiagent system in java agent development framework (JADE) via agent communication language messages. The performance of the proposed method is compared with existing load shedding allocation algorithms, i.e., proportional method, bankruptcy Talmud rule, and absolute contribution-based reward methods.

Lease Accounting: Back into the Past—A General Review of Different Theoretical Approaches

Purpose – For better mapping the path of lease accounting research, the purpose of this paper is to offer a general review of the existing different ways accounting literature has framed leasing operations in the book-keeping model and, especially, in the accounting equation.
 
 Design/methodology/approach – The literature reviewed consists of all the studies published in accounting academic journals available in the database ‘Business Source Premier’ from 1950 to July 2016 and presenting the word ‘lease’ or ‘leasing’ in the title, and the term ‘accounting’ in the main text. 
 
 Findings – The research has proved that while there is an overall agreement that, for accounting purposes, leasing has to be considered in the wider category of executory contracts, three main different theoretical approaches can be distinguished in literature: (i) the property right view; (ii) the firm commitment approach; and (iii) the risk and rewards approach.
 
 Originality/value – In order to fill the gap found in the literature and the non-existence of a study clarifying the main positions concerning the theoretical interpretation and the consequent accounting recognition of leasing in its predominantly financial and trilateral configuration, this paper makes an original contribution to the lease accounting research: providing a first organic reconstruction of the main academic studies dealing with this specific issue.

Lz-Transform Method and Markov Reward Approach for Flexible Manufacturing System Performance Evaluation

This article presents a performance evaluation of a flexible manufacturing system (FMS), in which the system and its components can have different performance levels ranging from perfect functioning to total failure. An original multistate transition process for the FMS’s main component, Computerized Numerical Control (CNC) machine tools, proposed in the current article and treated as an aging process because of its increasing failure rate is investigated. The Markov reward approach is utilized to analyze the availability of CNC machine tools and sensitivity analysis of each non-operational state to the machine’s downtime. To overcome the “state explosion problem” in the subsequent FMS performance evaluation process, an application of the Lz-transform method is proposed to determine the performance evaluation and availability of such aging multistate FMS. An illustrated case is presented to demonstrate the performance evaluation process. The results show that the suggested methods can be implemented in engineering decision making and construction of various aging multistate FMS related to demands, availability, and output performance.

The role of the nucleus accumbens in learned approach behavior diminishes with training.

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Markov Reward Approach and Reliability Associated Cost Model for Machine Tools Maintenance-Planning Optimization

This paper proposes a novel Reliability Associated Cost (RAC) model for machine tools throughout its lifetime that considers two different failure consequences, immediate failure and product rejections increase failure. A maintenance strategy of corrective maintenance combined with overhaul utilized to the maintenance activities of machine tools in the current paper. Markov reward approach is developed for computing of the costs incurred by both failure consequences and maintenance activities and system average availability throughout the machine tools life cycle. The Genetic Algorithm is used to find the optimal repair rates layout and overhaul moments that provide a minimal expected cost of system operation and maintenance actions and satisfies the desired availability requirement. A numerical example is presented in order to illustrate the approach and the results show that the proposed technique can significantly cut the RAC for machine tools.

The influence of oxytocin on risk‐taking in the balloon analogue risk task among women with bulimia nervosa and binge eating disorder

Previous theoretical models of bulimia nervosa (BN) and binge eating disorder (BED) have implicated cross-domain risk-taking behaviour as a significant maintenance factor in both disorders. The present study aimed to test this hypothesis by administering the Balloon Analogue Risk Task (BART) to 25 women with BN or BED and 27 healthy comparison women without a history of an eating disorder. Furthermore, we tested the effect of a divided dose of 64IU of oxytocin on risk-taking behaviour in the BART. Contrary to our hypothesis, women with BN or BED did not exhibit baseline differences in performance on the BART in the placebo condition (t=1.42, df=50, P=0.161, d=0.39). Oxytocin did not have a main effect on performance in the BART (F=0.01, df=1, P=.907, η2 partial <0.001); however, there was an interaction, such that participants in the BN/BED participant group, compared to the healthy comparison group, demonstrated safer behaviour on the BART in the oxytocin condition, but not in the placebo condition (F=4.29, df=1, P=0.044, η2 partial =0.082). These findings cast doubt on the common assumption that individuals with BN and BED exhibit greater risk-taking behaviour in all domains and add to the evidence that oxytocin plays a functional role in modulating behaviours that entail trade-offs between reward approach and risk in humans. We recommend that future dose-response studies investigate the effect of oxytocin on reward approach behaviour further in women with recurrent binge eating behaviour, as well as the clinical significance of this effect.

A Copula-based Markov Reward Approach to the Credit Spread in the European Union

ABSTRACTIn this paper, we propose a methodology based on piecewise homogeneous Markov chain for credit ratings and a multivariate model of the credit spreads to evaluate the financial risk in the European Union (EU). Two main aspects are considered: how the financial risk is distributed among the European countries and how large is the value of the total risk. The first aspect is evaluated by means of the expected value of a dynamic entropy measure. The second one is solved by computing the evolution of the total credit spread over time. Moreover, the covariance between countries’ total spread allows the understanding of any contagions in the EU. The methodology is applied to real data of 24 European countries for the three major rating agencies: Moody’s, Standard & Poor’s and Fitch. Obtained results suggest that both the financial risk inequality and the value of the total risk increase over time at a different rate depending on the rating agency. Moreover, the results indicate that the dependence structure is characterized by a strong correlation between most of the European countries.

PERFORMANCE MEASURES FOR THE TWO-NODE QUEUE WITH FINITE BUFFERS

We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds in order to bound performance measures of such random walks. The approximation scheme is developed in terms of a perturbed random walk in which the transitions along the boundaries are different from those in the original model and the invariant measure of the perturbed random walk is of product-form. We then apply this approximation scheme to a tandem queue and some variants of this model, for the case that both buffers are finite. The modified approximation scheme and the corresponding applications for a two-node queueing system in which only one of the buffers has finite capacity have also been discussed.

The future of canine glaucoma therapy.

Canine glaucoma is a group of disorders that are generally associated with increased intraocular pressure (IOP) resulting in a characteristic optic neuropathy. Glaucoma is a leading cause of irreversible vision loss in dogs and may be either primary or secondary. Despite the growing spectrum of medical and surgical therapies, there is no cure, and many affected dogs go blind. Often eyes are enucleated because of painfully high, uncontrollable IOP. While progressive vision loss due to primary glaucoma is considered preventable in some humans, this is mostly not true for dogs. There is an urgent need for more effective, affordable treatment options. Because newly developed glaucoma medications are emerging at a very slow rate and may not be effective in dogs, work toward improving surgical options may be the most rewarding approach in the near term. This Viewpoint Article summarizes the discussions and recommended research strategies of both a Think Tank and a Consortium focused on the development of more effective therapies for canine glaucoma; both were organized and funded by the American College of Veterinary Ophthalmologists Vision for Animals Foundation (ACVO‐VAF). The recommendations consist of (a) better understanding of disease mechanisms, (b) early glaucoma diagnosis and disease staging, (c) optimization of IOP‐lowering medical treatment, (d) new surgical therapies to control IOP, and (e) novel treatment strategies, such as gene and stem cell therapies, neuroprotection, and neuroregeneration. In order to address these needs, increases in research funding specifically focused on canine glaucoma are necessary.

Purchase Intention for Crowd-funded Milk Products with Integrated Photovoltaic Water Pumping Systems in China

In comparison with current financing mechanisms for renewable energy systems, crowd-funding financing mechanism offers a new potential source of financing with recent use of social media. Crowd-funding financing mechanism can also increases the social supports for renewable energy systems as users and investors turn to be more actively engaged in energy systems. As a new potential source of financing, crowd-funding mechanism has different forms, including donation, lending, equity and product reward approaches. In this paper, discrete choice model was used to explore whether crowd-funding financing with a novel sociotechnical product reward practice, has the attractions for potential customers to pay for a more sustainable milk product with distributed photovoltaic (PV) system. We empirically investigated the reward-base crowd funding with the specific integrated photovoltaic water pumping (PVWP) system in dairy milk production in China. 48 in-depth interviews were adopted for qualitative analysis of determinants of customer milk purchase decision. The ordered probit regression was employed with 357 online surveys to systematically estimate the purchase intention for the online-crowd-funding sustainable milk. Customer behaviours, environmental consciousness, and the individual socio-demographic factors were tested as potential explanatory variables. In the survey and depth interview samples, we found interviewees as potential customers showed strong purchase intentions to the crowd funding dairy milk for noticing milk quality and nutritious improvement, emission reduction and environmental benefits by the integrated PVWP system. In our findings of the regression results, the females, customers with young children or planning to have children were found with higher willing to purchase than other customers for crowd funding the sustainable dairy milk. The familiarity and popularity with online shopping and pre-sale purchase in China made customers more open and active towards pre-pay and crowd-funding mechanism.

Reward depending on public funds stimulates cooperation in spatial prisoner’s dilemma games

Prisoner’s dilemma (shortly, PD) games are studied on a square lattice, in which reward mechanisms are considered to stimulate cooperation. It is known to all that results vary with different reward methods. The tax mechanism, an effective tool to adjust the economy, inspires a reward approach where each player should pay corresponding taxes according to their payoff ranks to gather public funds, which is utilized to reward cooperators. There are three main reward levels: high intensity, middle intensity and low intensity. When total public funds keep relatively stable, the reward coverage is determined by the reward intensity. In other words, high intensity of reward is accompanied with narrow range and low intensity accompanies with wide range. Through the proposed model, whether the new reward mechanism can stimulate cooperation and what reward level is the optimum choice could be studied. Simulations reveal that this new mechanism is of great benefit to cooperation and it is noteworthy that low reward intensity with wide coverage has the biggest impact on cooperation.

Ethanol-assisted restoration of graphitic structure with simultaneous thermal reduction of graphene oxide

Fabrication of graphene via oxidation–reduction method is a rewarding approach in terms of scalability and cost of production. In this regard, restoration of the lattice defects presents in graphene oxide (GO) during the thermal reduction in the presence of a carbon source is one of the best alternatives. However, the mechanism of restoration is still unclear. This article presents the restoration in the presence of ethanol at a moderately high temperature by using a simple chemical vapor deposition setup, mainly focusing on the contribution of ethanol in the restoration with the increase in reduction time to postulate a possible mechanism. Raman spectroscopy, X-ray photoelectron spectroscopy, and for the first time atomic force microscopy along with Kelvin probe force microscopy were adopted to fulfill the objective.

Neural Activity in Ventral Medial Prefrontal Cortex Is Modulated More Before Approach Than Avoidance During Reinforced and Extinction Trial Blocks.

Ventromedial prefrontal cortex (vmPFC) is thought to provide regulatory control over Pavlovian fear responses and has recently been implicated in appetitive approach behavior, but much less is known about its role in contexts in which appetitive and aversive outcomes can be obtained and avoided, respectively. To address this issue, we recorded from single neurons in vmPFC while male rats performed our combined approach and avoidance task under reinforced and non-reinforced (extinction) conditions. Surprisingly, we found that cues predicting reward modulated cell firing in vmPFC more often and more robustly than cues preceding avoidable shock; in addition, firing of vmPFC neurons was both response (press or no-press) and outcome (reinforced or extinction) selective. These results suggest a complex role for vmPFC in regulating behavior and support its role in appetitive contexts during both reinforced and non-reinforced conditions.SIGNIFICANCE STATEMENT Selecting context-appropriate behaviors to gain reward or avoid punishment is critical for survival. Although the role of ventromedial prefrontal cortex (vmPFC) in mediating fear responses is well established, vmPFC has also been implicated in the regulation of reward-guided approach and extinction. Many studies have used indirect methods and simple behavioral procedures to study vmPFC, which leaves the literature incomplete. We recorded vmFPC neural activity during a complex cue-driven combined approach and avoidance task and during extinction. Surprisingly, we found very little vmPFC modulation to cues predicting avoidable shock, whereas cues predicting reward approach robustly modulated vmPFC firing in a response- and outcome-selective manner. This suggests a more complex role for vmPFC than current theories suggest, specifically regarding context-specific behavioral optimization.

Carbon gels with tuned properties for catalysis and energy storage

Mesoporous carbon materials are required for many applications, in order to avoid the diffusional limitations that occur with conventional activated carbons when processing large molecules or ions. The most rewarding approaches for the synthesis of mesoporous carbons involve sol–gel chemistry or the use of templates. Thus, carbon gels are obtained by polymerization of hydroxybenzenes and aldehydes by sol–gel processing, while ordered mesoporous carbons can be obtained from the same precursors in the presence of templates. In both cases, the textural properties of these nanostructured carbons can be easily adjusted by adequate selection of the synthesis variables. Their surface chemical properties can be modified by doping with heteroatoms and functionalization with surface groups. This can be done during the synthesis, or by subsequent treatments. The possibility of tuning both the texture and the surface chemistry is essential for the development of high-performance carbon materials that meet the requirements of the targeted applications. Recent advances and challenges related to the preparation of nanostructured carbon gels will be reviewed, together with relevant examples of their application in catalysis and energy storage.

Evidence of structure and persistence in motivational attraction to serial Pavlovian cues.

Sign-tracking is a form of autoshaping where animals develop conditioned responding directed toward stimuli predictive of an outcome even though the outcome is not contingent on the animal's behavior. Sign-tracking behaviors are thought to arise out of the attribution of incentive salience (i.e., motivational value) to reward-predictive cues. It is not known how incentive salience would be attributed to serially occurring cues, despite cues often occurring in a sequence in the real world as reward approaches. The experiments presented here demonstrate that reward-proximal cue responding is not altered by the presence of a distal reward cue (Experiment 1), and similarly that reward-distal cue responding which animals favor, is not altered by the presence of a reward-proximal cue (Experiment 2). Extinction of reward-proximal cues after training of the serial sequence leads to a generalized reduction in lever responding (Experiment 3). Together, we show that both Pavlovian serial lever cues acquire motivational value. These experiments also provide support to the notion that sign-tracking responses are insensitive to changes in outcome value, and that responding to serial cues creates a distinct context for outcome value.

Availability estimation of a multi-state wind farm in fuzzy environment

ABSTRACTWind is one of the fastest growing renewable energy resources in the electric power system. Availability of wind energy is volatile in nature due to the stochastic behavior of wind speed and non-linear variation of the wind power curve of wind turbine generator. Because of this impression and uncertainty, the availability estimation of wind power has become a challenging issue. In this paper, Markov Fuzzy Reward technique has been proposed for finding out the reliability of wind farm by assessing the availability of wind power. According to this technique, availability of the wind power has been estimated considering wind farm and demand both as a multi-state system. In addition to the availability, different reliability indices such as the number of absolute failures, mean time to deficiency, and probability of failures of a wind farm have been assessed in a time horizon, which can provide useful information for the power system planner at wind farm installing stage. A comparison of this study reveals the efficacy of the proposed Markov Fuzzy Reward approach over the conventional Markov Reward approach.

REWARD AND PUNISHMENT PROJECT APPROACH: Model development and pilot study

Tax sanctions are vital for tax system to ensure tax laws to be obeyed. Despite the importance of tax sanctions, low feedbacks and law enforcement may limit the effectiveness of a tax system and reducing trust on the government. Therefore, the role of public officials in law enforcement is important. However, studies related to the role of tax sanction and public official are dearth. One of them is the study focusing taxpayers employed in public sectors. This study highlighted the tax sanction as the lowest assessment and effect on tax compliance. Then, the study conducted recommended the reward and punishment as a notion of other form of tax sanctions. This paper focuses on the development of model of reward and punishment based on the Project Design Matrix (PDM) by the steps involving problem analysis, objective analysis, generating project design matrix. The developed project provides a foundation for government and for future studies regarding reward and punishment in the tax sanction framework.

“I might be shot at!” exploring the drivers to work in hostile environments using an intelligent careers perspective

Purpose The purpose of this paper is to explore the factors which influence individuals to take up international assignments in hostile environments (HEs). Using an intelligent careers (IC) perspective, an expanded framework of expatriation drivers to work in hostile contexts is developed that comprises individual, organizational and location-specific factors. In addition, the understanding of career capital acquisition and transfer is refined. Design/methodology/approach A “deviant” case study method to challenge the underlying assumptions of career capital maximization and transfer in global careers is used. To investigate the case, 25 individuals in an international development organization who had to decide whether to work in HEs were interviewed. Findings Five insights into decision drivers and career capital effects associated with postings to HEs are presented. These span all three levels of individual, organizational and location-specific decision factors. Research limitations/implications Due to the case study approach, the usual limitations of qualitative case-based research with respect to generalizability apply. In the conclusions three theoretical implications for the IC framework with respect to career capital acquisition, utilization and temporal effects are outlined. Practical implications A range of practical implications in relation to the selection, talent management, performance and reward approaches as well as repatriation and family considerations in global mobility are explored. Social implications The insights help organizations to design global mobility policies for HEs. In addition, individuals and their families benefit from greater clarity of global mobility drivers in the context of high risks. Originality/value The drivers of individuals to accept assignments to HEs are under-researched. This paper operationalizes and applies a holistic decision to work abroad framework, expands the literature on of the motivations of individuals and develops valuable insights to nuance the IC framework.

The effects of ethanol on diverse components of choice in the rat: reward discrimination, preference and relative valuation.

Alcohol consumption impairs judgment and choice. How alcohol alters these crucial processes is primarily unknown. Choice can be fractionated into different components including reward discrimination, preference and relative valuation that can function together or in isolation depending upon diverse factors including choice context. We examined the diverse components and contextual effects by analyzing the effects of alcohol drinking on choice behavior in a task with a reduced level of temporal and spatial constraints. Rats were trained to drink 10% ethanol during 6weeks of behavior testing using a combined sucrose-fade and two-bottle free-choice procedure. Two different sucrose pellet outcomes (e.g., constant vs. variable) were presented each week to examine the impact of voluntary drinking on reward-based decision-making. Behavioral contexts of single option, free choice and extinction were examined for each outcome set. Comparisons were made between alcohol and control groups and within the alcohol group over time to inspect choice profiles. Between-group results showed alcohol drinking animals expressed altered place preference and modified sucrose reward approach latencies. The within-group profile showed that alcohol drinking animals can express adequate reward discrimination, preference and incentive contrast during free choice. All of these components were significantly reduced during the context of extinction. Control animals were also impacted by extinction but not as severely. The findings point to a need for a greater focus on the context and the diverse components of choice when examining external and internal factors influencing decision-making during alcohol or other substance of abuse exposure.