Superior Reward Research Articles

Offline reinforcement learning based feeding strategy of ethylene cracking furnace

The feeding process of the ethylene cracking furnace necessitates the synchronized adjustment of multiple controlled factors. The process mainly relies on operators to do it manually, which is burdensome and may lead to significant variations in coil out temperature (COT) due to the differing expertise of operators. This paper proposes a method for learning the feeding strategy of the ethylene cracking furnace using offline reinforcement learning. The agent learns and optimizes the operating strategy directly from datasets, eliminating the need for sophisticated process simulator modeling. In addition, the advantage function is incorporated into the Twin Delayed Deep Deterministic Behavioral Cloning (TD3BC) algorithm, which enables the agent to acquire more effective operational experience. The proposed method is initially evaluated using benchmark datasets. Further, the proposed method is validated through comparative experiments on a feeding process validation model, demonstrating superior rewards and outperforming manual operating experience as well as other offline reinforcement learning methods.

Psychometric evaluation of a new instrument to hookah tobacco smoking-related factors in the Iranian female university students based on the Extended Parallel Process Model.

The higher prevalence of hookah tobacco smoking (HTS) has become worrisome in women. Thus, it is essential to identify and measure the perceptions associated with this unhealthy behavior as well as the type of response to risk messages to design the effective programs. This study aimed to psychometric evaluation of a new instrument, named the Extended Parallel Process Model-Hookah Tobacco Smoking Questionnaire (EPPM-HTSQ), for female university students. This methodological study was conducted in 2019-2020 in central and western Iran. The first version of the instrument was developed with 97 items. Face, Content, and construct validity methods were used to assess the EPPM-HTSQ validity. The data viability for factorability was guided through Kaiser-Meyer-Olkin (KMO) measure of sample adequacy (Values >0.7) and Bartlett's test of Sphericity (P < 0.05). The Exploratory Factor Analysis (EFA) method with varimax rotation and SPSS software was performed to identify the main factors of the questionnaire and interpretable factors. The internal consistency and external reliability were determined. The test-retest was used for evaluating the stability of tools. During the assessment of the face and content validity 71 items remained in the questionnaire. EFA led to the extraction of nine dimensions: "Focusing on superior rewards," "Response Efficacy," "Perceived physical threat," "Perceived social threat," "Self-efficacy," "Fear control," "Attitude," "Fear," and "Intention" Kaiser-Meyer-Olkin (KMO) test (0.957) and Bartlett's test (P < 0.001). Cronbach's alpha (0.871-0.951) and ICC (0.985-0.998) were approved for scale dimensions. Important findings of the present study were the development and validation of the EPPM-HTSQ for measuring EPPM variables in relation to HTS; introducing a new variable "focusing on superior rewards"; as well as, the new classification of the concept of "perceived threat." The final version of the EPPM-HTSQ is a valid and reliable tool, but it is suggested to be re-evaluated in other studies with different populations and sample sizes.

Trajectory Optimisation for Cooperative Target Tracking with Passive Mobile Sensors

The paper considers the problem of tracking a moving target using a pair of cooperative bearing-only mobile sensors. Sensor trajectory optimisation plays the central role in this problem, with the objective to minimize the estimation error of the target state. Two approximate closed-form statistical reward functions, referred to as the Expected Rényi information divergence (RID) and the Determinant of the Fisher Information Matrix (FIM), are analysed and discussed in the paper. Being available analytically, the two reward functions are fast to compute and therefore potentially useful for longer horizon sensor trajectory planning. The paper demonstrates, both numerically and from the information geometric viewpoint, that the Determinant of the FIM is a superior reward function. The problem with the Expected RID is that the approximation involved in its derivation significantly reduces the correlation between the target state estimates at two sensors, and consequently results in poorer performance.

Consummatory succesive positive contrast produced by the downshift of an aversive solution in infant rats.

Subjects trained in successive positive contrast are usually given an appetitive stimulus of relatively low quality during a pre-shift, followed by exposure to a significantly greater quality of the same stimulus. Enhanced responsiveness to the high-quality stimulus during the post-shift phase, compared to a control group that receives the superior reward in both phases, is taken as an index of successive positive contrast. Successive positive contrast reports are rare, probably due to performance limitations inherent to the experimental protocols available. We exposed infant rats (14 days old at the start of training) to .1% or .01% quinine during 4, 10 min, trials (pre-shift phase). All animals were then given two trials of exposure to .01% quinine (post-shift phase). During the pre-shift the level of intake was greater in pups stimulated with the relatively less aversive .01% quinine solution. These animals also exhibited, compared to those stimulated with .1% quinine, lower emission of the aversive response paw treading. During the post-shift phase, the group that had been exposed to .1% quinine exhibited significantly greater intake of .01% quinine, along with a reduction in the emission of paw treading and an enhancement in paw licking, an ingestive, appetitive response. Altogether, the evidence is suggestive of the emergence of consummatory successive positive contrast during the second week of life of the rat. To our knowledge, this is the first evidence of positive contrast using an aversive solution.

Inequity aversion strategies between marmosets are influenced by partner familiarity and sex but not by oxytocin

Cooperation among individuals depends, in large part, on a sense of fairness. Many cooperating non-human primates (NHPs) show inequity aversion, (i.e., negative responses to unequal outcomes), and these responses toward inequity likely evolved as a means to preserve the advantages of cooperative relationships. However, marmosets (Callithrix spp.) tend to show little or no inequity aversion, despite the high occurrence of prosociality and cooperative-breeding in callitrichid monkeys. Oxytocin [OXT] has been implicated in a wide variety of social processes, but little is known about whether OXT modulates inequity aversion toward others. We used a tray pulling task to evaluate whether marmosets would donate superior rewards to their long-term pairmate or an opposite-sex stranger following OXT, OXT antagonist, and saline treatments. We found that marmosets show inequity aversion, and this inequity aversion is socially- and sex-specific. Male marmosets show inequity aversion toward their pairmates but not strangers, and female marmosets do not show inequity aversion. OXT treatments did not significantly influence inequity aversion in marmosets. While OXT may modulate prosocial preferences, the motivations underlying cooperative relationships, such as inequity aversion, are multifaceted. More research is needed to evaluate the evolutionary origins, biological processes, and social contexts that influence complex phenotypes like inequity aversion. Inequity aversion can differ within species in important and distinct ways including between individuals who do and do not share a cooperative relationship. Overall, these findings support the view that inequity aversion is an important behavioural strategy for the maintenance of cooperative relationships.

Chimpanzees (Pan troglodytes) Flexibly Adjust Their Behaviour in Order to Maximize Payoffs, Not to Conform to Majorities

Chimpanzees have been shown to be adept learners, both individually and socially. Yet, sometimes their conservative nature seems to hamper the flexible adoption of superior alternatives, even to the extent that they persist in using entirely ineffective strategies. In this study, we investigated chimpanzees’ behavioural flexibility in two different conditions under which social animals have been predicted to abandon personal preferences and adopt alternative strategies: i) under influence of majority demonstrations (i.e. conformity), and ii) in the presence of superior reward contingencies (i.e. maximizing payoffs). Unlike previous nonhuman primate studies, this study disentangled the concept of conformity from the tendency to maintain one’s first-learned strategy. Studying captive (n=16) and semi-wild (n=12) chimpanzees in two complementary exchange paradigms, we found that chimpanzees did not abandon their behaviour in order to match the majority, but instead remained faithful to their first-learned strategy (Study 1a and 1b). However, the chimpanzees’ fidelity to their first-learned strategy was overridden by an experimental upgrade of the profitability of the alternative strategy (Study 2). We interpret our observations in terms of chimpanzees’ relative weighing of behavioural options as a function of situation-specific trade-offs. More specifically, contrary to previous findings, chimpanzees in our study abandoned their familiar behaviour to maximize payoffs, but not to conform to a majority.

Capuchin monkeys (Cebus apella) fail to show inequality aversion in a no-cost situation

Although humans show robust equality concerns across a variety of situations, there is ongoing debate regarding the extent to which any nonhuman species is inequality averse. In the current research, we test nonhuman primates' reactions to conspecifics receiving equal and unequal payoffs using a “no-cost” method in which subjects can respond to inequality without rejecting food. Specifically, we gave capuchin monkeys (Cebus apella) the opportunity to trade with one of two experimenters, each of whom offered the subject an identical reward, but had different histories of trading with the subject and a conspecific partner. An “equal” experimenter had previously given a conspecific the same reward that the subject had received, whereas the other experimenter was either an “advantageous trader” for the subject (giving the conspecific an inferior reward) or a “disadvantageous trader” for the subject (giving the conspecific a superior reward). By offering subjects a choice between experimenters, we removed several competing demands that may have masked the expression of robust equality preferences in previous studies. Even though there was no cost associated with expressing an equality preference, we found no evidence that capuchins differentiated between equal and unequal experimenters.

The role of energetic value in dynamic brain response adaptation during repeated food image viewing

The repeated presentation of simple objects as well as biologically salient objects can cause the adaptation of behavioral and neural responses during the visual categorization of these objects. Mechanisms of response adaptation during repeated food viewing are of particular interest for better understanding food intake beyond energetic needs. Here, we measured visual evoked potentials (VEPs) and conducted neural source estimations to initial and repeated presentations of high-energy and low-energy foods as well as non-food images. The results of our study show that the behavioral and neural responses to food and food-related objects are not uniformly affected by repetition. While the repetition of images displaying low-energy foods and non-food modulated VEPs as well as their underlying neural sources and increased behavioral categorization accuracy, the responses to high-energy images remained largely invariant between initial and repeated encounters. Brain mechanisms when viewing images of high-energy foods thus appear less susceptible to repetition effects than responses to low-energy and non-food images. This finding is likely related to the superior reward value of high-energy foods and might be one reason why in particular high-energetic foods are indulged although potentially leading to detrimental health consequences.

Giving is self-rewarding for monkeys.

Helping and sharing among humans is often motivated by empathy and accompanied by a sense of satisfaction. To determine whether similar self-rewarding mechanisms may underpin assistance among nonhuman primates, eight female brown capuchin monkeys (Cebus apella) underwent testing in a simple choice paradigm. Paired with a partner, subjects could select either a "selfish" option that rewarded only themselves, or a "prosocial" option that rewarded both of them. Subjects systematically favored the prosocial option provided their partner was a) familiar, b) visible, and c) receiving rewards of equal value. Prosocial tendencies increased with social closeness, being lowest toward strangers and highest toward kin. That the monkeys understood the options was suggested by greater orientation to the partner during prosocial than selfish choices. Prosocial preferences were reduced by inequity, when the partner received a superior reward. If the view between both monkeys was blocked, choices became strikingly selfish. Thus, under certain conditions, delivering benefits to others seems gratifying to nonhuman primates.

Partial support from a nonreplication: Comment on Roma, Silberberg, Ruggiero, and Suomi (2006).

A purported replication by P. G. Roma, A. Silberberg, A. M. Ruggiero, and S. J. Suomi of the authors' previous study claims to contradict their finding that capuchin monkeys (Cebus apella) refuse to exchange with an experimenter if their partner receives a superior reward. Roma et al. used no exchange task, however, or any other task. Roma et al. offered frustration as explanation of their findings, yet failed to statistically prove that the effect of frustration is stronger than that of inequity. They also misrepresented the dependent measure of the authors' study. Reanalysis of the authors' own data indicated no role of frustration, that is, no effect of previous experience with a superior reward. The authors conclude that Roma et al.'s study is not a replication and does not disprove the authors' findings.

Which Risks Have Been Best Rewarded?

An empirical study examines the consistency of rewards for bearing various types of risks in U.S. asset markets between 1985 and early 2002. Bearing duration risk and equity market risk was amply rewarded, while bearing long-dated credit risk buying credit-risky bonds versus governments realized puny average profits. Bearing short-dated credit risk gave the most consistent profits of all static overweight strategies, with an information ratio near one. Wide break-even spread cushions may be one explanation for the superior reward for risk at short maturities. More fundamentally, market segmentation seems to be the main explanation, because the most consistent profit opportunities involve switching from Treasuries to top-rated non-government bonds.

Subordinate imitation of supervisor behavior: The role of modeling in organizational socialization

Social Learning Theory predictions regarding imitation of work behavior models with specific social characteristics were tested. Data were collected from 141 pairs of lower-level supervisors (subordinates) and their direct superiors in seven organizations. All participants described their own supervisory leadership style, and subordinates also described the competence, reward power, and success within the organization of their superiors, their own self-esteem, and the behavior reward contingencies of their organizations. Results provided support for the Social Learning Theory predictions. The degree of behavior similarity displayed by subordinates and their direct superiors was positively correlated with subordinates' perceptions of their superiors' success and competence. However, behavior similarity was unrelated to perceptions of superior reward power. Further analyses indicated that the correlations between perceived superior characteristics and behavior similarity were significantly moderated by subordinate self-esteem and that behavior reward expectancies could be considered to be intervening variables only when analyzing the influence of model success. Results are discussed in terms of a cognitive motivational framework for analyzing socialization processes.

Toward a Refinement of the "Marginal Man" Concept

Tj~HE phenomenon of marginality, though fairly widespread in the modern world, J has received scant attention from social scientists.' Marginality, I would suggest, refers to a social situation characterized by the following: 1. Two cultures (or subcultures) are in lasting contact. 2. One of them is dominant in terms of power and reward potential. This is the non-marginal culture of the two. Its members are not particularly influenced by or attracted to the other, the marginal culture. 3. The boundaries between the two are sufficiently permeable for the members of the marginal culture to internalize the patterns of the dominant culture as well as that of their own. 4. These patterns, in their entirety, cannot be easily harmonized. 5. Having acquired the goals of the non-marginal culture, members of the marginal group are pulled by the promise of the greater rewards offered. 6. The barriers between the two tend to be hardened by discrimination from the one side, and by pressure against betrayal from the other. 7. Marginality acquires particular intensity when the clash persists through more than one generation. On the basis of this characterization of marginality, we can distinguish it from other foci of attention within the general field in which it lies, namely, culture contacts-the study of the laws of development of cultures and social structures under the impact of inter-societal contact. The study of marginality is different from that of: (1) the diffusion of culture between relatively independent societies; (2) the relations between groups in contact with relatively impermeable boundaries, e.g., situations of classical imperialism; and (3) the relatively rapid acculturation and assimilation of individuals orfragmented groupings, emigrants from one culture to another which is hospitable and accepting, e.g., old immigration to the United States. Marginality is also to be distinguished from alienation and anomie. It is often said that our society is characterized by these, by no one quite feeling that he belongs anywhere. This is different from the state where one has ties to two partially incompatible societies, and does not wholly belong to either. One can be anomic without being marginal, and marginal without being anomic. It is also frequently argued that modern society is nought but a complex of subcultures, and that, inasmuch as each of us is a member of a class, ethnic, religious, and possibly of other subcultures whose bounds rarely coincide, we are all marginal. Multiple subculture membership can, indeed, be marginal, if there is a conflict between the values and patterns of the subcultures involved, and a sense of belonging to both simultaneously is difficult to achieve. To the extent that this is so, marginality is not to be seen as a phenomenon limited to immigrant groups and their children. I would also call attention to that part of the above characterization of marginality which stresses the superior reward potential of the nonmarginal group. Without this differential, that is, were there no goals which could better or only be realized in the dominant group, members of the marginal group would scarcely be pulled away from it. * Expanded version of a paper read at the annual meeting of the American Sociological Society, September 1955. Appreciation is expressed to the Department of Scientific Research of the American Jewish Committee for support of the study upon which this paper is based. 1 Of the seven published articles in sociological journals in the last dozen years which deal with marginality, only two derive from empirical studies. The others are discussion papers. In 1929 Karl Mannheim placed his hope on the 'socially unattached intelligentsia' as those who would evolve a progressively comprehensive and rational understanding of Freedom, Power and Democratic Planning (New York: Oxford University Press, 1950), p. ix. Since intellectuals are marginal men, studying marginality becomes a matter of self-study, a difficult and uncomfortable affair. This may, in part, explain the neglect of this aspect of reality.

Work and Needs of the Smithsonian Institution

THE Smithsonian Institution is appealing to the American public for an"addition of ten million dollars towards its endowment. The grounds of this appeal were put in a very forcible and interesting way by Mr. Austin Hobart Clark, in an address to the American Association for the Advancement of Science, at Kansas City, on January i. Readers of NATURE know something of the Smithsonian and what it has done. Its general policy has been to encourage research along all lines that do not receive adequate support from other sources, but never to interfere with work done under other auspices. In this way it has led to the establishment of ten technical bureaux, which now proceed as Government establishments, though in close touch with, and sometimes under the supervision of, the Smithsonian. The fields in which the Institution is desirous of expanding are mainly those of astrophysics, oceanography, entomology, palaeontology, anthropology and archaeology, animal diseases, botany, zoology, intensive exploration, and publications. While far from unmindful of the economic importance of these varied activities, the Smithsonian recognises that the vast extension of applied science and its superior rewards are attracting the best of the younger generation, so that the stream of those to whom we should otherwise look for the elaboration of new ideas and theories is being deflected elsewhere. It will therefore be the aim of the Smithsonian in the future, even more than in the past, to promote research in pure science.