Partial Reinforcement Research Articles

Pavlovian conditioning under partial reinforcement: The effects of nonreinforced trials versus cumulative conditioned stimulus duration.

A core feature of associative models, such as those proposed by Allan Wagner (Rescorla & Wagner, 1972; Wagner, 1981), is that conditioning proceeds in a trial-by-trial fashion, with increments and decrements in associative strength occurring on each occasion that the conditioned stimulus (conditional stimulus, or CS) is present either with or without the unconditioned stimulus (US). A very different approach has been taken by theories that assume animals continuously accumulate information about the total length of time spent waiting for the US both during the CS and in the absence of the CS (e.g., Gallistel & Gibbon, 2000). Here we describe 3 experiments using within-subject designs that tested trial-based and time-accumulation accounts of the acquisition of conditioned responding using magazine approach conditioning in rats. We found that responding was affected by the total (cumulative) duration of exposure to the CS without the US rather than the number of trials on which the CS occurred without the US. We also found that exposure to the CS without the US had the same effect on conditioning whether that exposure occurred shortly (60 s) before each CS-US pairing or whether it occurred long (240 s) before each pairing. These findings are more consistent with time-accumulation models of conditioning than trial-based models like the Rescorla-Wagner model and Wagner's (1981) sometimes opponent process model. We discuss these findings in relation to other evidence that favors trial-based models rather than time-accumulation models. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Adaptive Partial Reinforcement Learning Neural Network-Based Tracking Control for Wheeled Mobile Robotic Systems

In this paper, a dynamic model of a wheeled mobile robotic (WMR) system with coupled control input is developed, which will increase the complexity of its tracking control with time-varying advance angle. To deal with this problem, a partial reinforcement learning neural network (PRLNN)-based tracking algorithm is proposed for the WMR systems. The main contributions of the PRLNN adaptive tracking control method is that it is the first control method to introduce the PRLNN adaptive control to the WMR system, which determines to solve the WMR tracking control with the time-varying advance angle. The critic neural network (NN) and action NN adaptive laws for the decoupled controllers are designed using the standard gradient-based adaptation method. According to the Lyapunov stability analysis theorem, the uniform ultimate boundedness of all signals in the WMR system can be guaranteed with the design parameters chose properly, and the tracking error converge to a small compact set nearby zero. A numerical simulation is presented to verify the effectiveness of the proposed control algorithm.

Investigation on the electrochemical and mechanical performance of CFRP and steel-fiber composite bar used for impressed current cathodic protection anode

Impressed current cathodic protection (ICCP) has been proved to be an effective method to inhibite steel corrosion. Combined with ICCP technique, a new system has been proposed in this paper with carbon fiber reinforced polymer (CFRP) as dual-functional material: auxiliary anode and partial reinforcement simultaneously. The electrochemical and mechanical performances of CFRP in the new ICCP system were investigated experimentally with two kinds of reinforcement bar: CFRP bar and steel-fiber composite bar (SFCB). First, the accelerated anodic polarization test and electrochemical impedance spectroscopy (EIS) measurement were carried out for CFRP bar and SFCB to verify the electrochemical performances. Then, the mechanical performance and underlying degradation mechanism were identified for CFRP bar and SFCB based on the uniaxial tensile test and scanning electron microscopy (SEM). Finally, the service life of CFRP bar and SFCB has been evaluated to confirm the fesibility of the proposed system in practical engineering. Results showed that CFRP bar and SFCB possess excellent electrical conductivity and low electrical resistance as the anode in ICCP system. Compared with SFCB, CFRP bar is more promising as an anode material with a stronger polarization resistance capability, an acceptable degradation of the mechanical property and a longer service life. With a current density of 20 mA/m2, the service life was evaluated as 62 years for CFRP bar while 7 years for SFCB. To improve the application of SFCB in ICCP system effectively, some modifications are necessary, such as replacing the inner steel bar with stainless steel bar or adding a protective layer between the steel bar and fiber layer. Furthermore, the proposed system shows great potential for marine RC structures and the comprehensive study of this system will be carried out in future work.

Learning mechanisms in nocebo hyperalgesia: the role of conditioning and extinction processes.

Nocebo hyperalgesia is a clinically relevant phenomenon and may be formed as a result of associative learning, implemented by classical conditioning. This study explored for the first time distinct nocebo conditioning methods and their consequences for nocebo attenuation methods. Healthy participants (N = 140) were recruited and randomized to the following nocebo hyperalgesia induction groups: conditioning with continuous reinforcement (CRF), conditioning with partial reinforcement (PRF), and a sham-conditioning control group. In the attenuation phase, counterconditioning was compared with extinction. During induction, participants experienced increased thermal pain in 100% of nocebo trials in the CRF groups, while in only 70% of nocebo trials in the PRF groups. During evocation, pain stimulation was equivalent across all trials. During attenuation, pain stimulation was decreased on nocebo trials relative to control trials for the counterconditioning groups, while pain remained equivalent across all trials for the extinction groups. Results showed that both PRF and CRF significantly induced nocebo hyperalgesia, but CRF was a more potent nocebo induction method, as compared to PRF. Counterconditioning was more effective than extinction in attenuating nocebo hyperalgesia. Neither CRF nor PRF resulted in resistance to extinction. However, compared with CRF, conditioning with PRF resulted in more resistance to counterconditioning. These findings demonstrate that the more ambiguous learning method of PRF can induce nocebo hyperalgesia and may potentially explain the treatment resistance and chronification seen in clinical practice. Further research is required to establish whether attenuation with counterconditioning is generalizable to clinical settings.

Longitudinal changes of resting-state functional connectivity of amygdala following fear learning and extinction

Altered functional connectivity of the amygdala has been observed in a resting state immediately after fear learning, even one day after aversive exposure. The persistence of increased resting-state functional connectivity (rsFC) of the amygdala has been a critical finding in patients with stress and anxiety disorders. However, longitudinal changes in amygdala rsFC have rarely been explored in healthy participants. To address this issue, we studied the rsFC of the amygdala in two groups of healthy volunteers. The control group participated in three fMRI scanning sessions of their resting state at the first visit, one day, and one week later. The experimental group participated in three fMRI sessions on the first day: a resting state before fear conditioning, a fear extinction session, and a resting state immediately after fear extinction. Furthermore, this group experienced scanning after one day and week. The fear-conditioning paradigm consisted of visual stimuli with a distinct rate of partial reinforcement by electric shock. During the extinction, we presented the same stimuli in another sequence without aversive pairing. In the control group, rsFC maps were statistically similar between sessions for the left and right amygdala. However, in the experimental group, the increased rsFC mainly of the left amygdala was observed after extinction, one day, and one week. The between-group comparison also demonstrated an increase in the left amygdala rsFC in the experimental group. Our results indicate that functional connections of the left amygdala influenced by fear learning may persist for several hours and days in the human brain.

Past outcomes and time flexibly exert joint control over midsession reversal performance in the rat

In a midsession reversal task, subjects choose between two stimuli on every trial; only responses to one stimulus are reinforced. Halfway throughout the session, contingencies are reversed: previously reinforced responses are now extinguished and vice versa. Both, the outcome of the previous trial and the time elapsed since the beginning of the session, may predict the availability of reinforcement and determine choice. Thus, this task has typically been used to study cognitive flexibility and the temporal organization of behavior. This study assessed how past outcomes and time interact for behavioral control when each cue predicts the availability of reinforcement to a different extent. Eight rats were trained in four variations of the midsession reversal task differing in the reliability of outcomes and time as predictors of the reinforced response. We manipulated the reliability of the outcomes by providing either continuous or partial reinforcement, and the reliability of time by fixing the moment of reversal (middle of the session) or making the reversal unpredictable (semi-random trial). Results suggest that behavioral control alternates between outcomes and time according to the relative reliability of each cue. Model simulations show that outcomes and time may jointly determine behavior, and that momentary reinforcement rate may determine their relative influence.

Characterization of Glass/Jute Hybrid Fibre Reinforced Epoxy Composite for Axial Flow Fan Blade

Traditionally axial flow fan blades were manufactured by using metal alloys, but with the increase in blade size the blades grew heavier resulting in more vibrations during its working. It is impossible to manufacture the metallic blades to close tolerances making it uneconomical. Due to this dimensional instability, the metallic impeller fan blades are instead fabricated with glass fiber reinforced plastics (GFRP). To reduce the after effects of GFRP blades on the environment necessitates the partial reinforcement of natural fibre. This paper deals with the characterization of GFRP blade material with partial Jute reinforcements i.e., glass to jute (95:5) weight percentage and estimating the properties like tensile strength, bending strength, shear strength, fracture toughness. The conventional GFRP blade material has 14 layers of glass fibers in various forms. By placing jute layer at various positions in GFRP blade material the mechanical strengths were estimated and compared with conventional one.

The partial reinforcement extinction effect depends on learning about nonreinforced trials rather than reinforcement rate.

Conditioned responding extinguishes more slowly after partial (inconsistent) reinforcement than after consistent reinforcement. This Partial Reinforcement Extinction Effect (PREE) is usually attributed to learning about nonreinforcement during the partial schedule. An alternative explanation attributes it to any difference in the rate of reinforcement, arguing that animals can detect the change to nonreinforcement more quickly after a denser schedule than a leaner schedule. Experiments 1a and 1b compared extinction of magazine responding to a conditioned stimulus (CS) reinforced with 1 food pellet per trial and a CS reinforced with 2 pellets per trial. Despite the difference in reinforcement rate, there was no reliable difference in extinction. Both experiments did demonstrate the conventional PREE comparing a partial CS (50% reinforced) with a consistent CS. Experiments 2 and 3 tested whether the PREE depends specifically on learning about nonreinforced trials during partial reinforcement. Rats were trained with 2 CS configurations, A and AX. One was partially reinforced, the other consistently reinforced. When AX was partial and A consistent, responding to AX extinguished more slowly than to A. When AX was consistent and A was partial, there was no difference in their extinction. Therefore, pairing X with partial reinforcement allowed rats to show a PREE to AX that did not generalize to A. Pairing A with partial reinforcement meant that rats showed a PREE to A that generalized to AX. Thus, the PREE depends on learning about nonreinforced trials during partial reinforcement and is not because of any difference in per-trial probability of reinforcement. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The importance of trials.

Many theories of conditioning describe learning as a process by which stored information about the relationship between a conditioned stimulus (CS) and unconditioned stimulus (US) is progressively updated upon each occasion (trial) that the CS occurs with, or without, the US. These simple trial-based descriptions can provide a powerful and efficient means of extracting information about the correlation between 2 events, but they fail to explain how animals learn about the timing of events. This failure has motivated models of conditioning in which animals learn continuously, either by explicitly representing temporal intervals between events or by sequentially updating an array of associations between temporally distributed elements of the CS and US. Here, I review evidence that some aspects of conditioning are not the consequence of a continuous learning process but reflect a trial-based process. In particular, the way that animals learn about the absence of a predicted US during extinction suggests that they encode and remember trials as single complete episodes rather than as a continuous experience of unfulfilled expectation of the US. These memories allow the animal to recognize repeated instances of nonreinforcement and encode these as a sequence that, in the case of a partial reinforcement schedule, can become associated with the US. The animal is thus able to remember details about the pattern of a CS's reinforcement history, information that affects how long the animal continues to respond to the CS when all reinforcement ceases. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

A flexible and generalizable model of online latent-state learning.

Many models of classical conditioning fail to describe important phenomena, notably the rapid return of fear after extinction. To address this shortfall, evidence converged on the idea that learning agents rely on latent-state inferences, i.e. an ability to index disparate associations from cues to rewards (or penalties) and infer which index (i.e. latent state) is presently active. Our goal was to develop a model of latent-state inferences that uses latent states to predict rewards from cues efficiently and that can describe behavior in a diverse set of experiments. The resulting model combines a Rescorla-Wagner rule, for which updates to associations are proportional to prediction error, with an approximate Bayesian rule, for which beliefs in latent states are proportional to prior beliefs and an approximate likelihood based on current associations. In simulation, we demonstrate the model’s ability to reproduce learning effects both famously explained and not explained by the Rescorla-Wagner model, including rapid return of fear after extinction, the Hall-Pearce effect, partial reinforcement extinction effect, backwards blocking, and memory modification. Lastly, we derive our model as an online algorithm to maximum likelihood estimation, demonstrating it is an efficient approach to outcome prediction. Establishing such a framework is a key step towards quantifying normative and pathological ranges of latent-state inferences in various contexts.

Discrete and Finite Element Models for the Analysis of Unreinforced and Partially Reinforced Masonry Arches

In this work the behavior of masonry arches, without reinforcement and with partial reinforcement, is investigated by means of three different numerical models. The first one is a Discrete Element model based on rigid blocks, and elastic-plastic interfaces; the second one is a standard heterogeneous Finite Element Model, which is adopted for a detailed micro-modelling of arch voussoirs, joints, and reinforcements. The third model is analytic-numerical, and it is adopted for validating the other numerical results. The aim of the work is the comparison and validation of the numerical Finite and Discrete Element models for the correct simulation of masonry arch behavior, together with the evaluation of the effectiveness of these models in simulating the behavior of the partially reinforced arch.

Reinforcement Contingency Learning in Children with ADHD: Back to the Basics of Behavior Therapy

Reinforcement deficits in ADHD may affect basic operant learning processes relevant for Behavioral Treatment. Behavior acquired under partial reinforcement extinguishes less readily after the discontinuation of reinforcement than behavior acquired under continuous reinforcement, a phenomenon known as the Partial Reinforcement Extinction Effect [PREE], which has great relevance for the emergence of behavioral persistence. The present study examined acquisition and extinction of operant responding under partial and continuous reinforcement in children with and without ADHD. In addition, we evaluated the effectiveness of gradual stretching the reinforcement rate during acquisition for remedying potential acquisition or extinction deficits under partial reinforcement in ADHD. In an operant learning task designed to mimic the task confronted by an animal in a Skinner box, 62 typically developing and 49 children with ADHD (age: 8–12) were presented with a continuous, partial or gradually stretching reinforcement scheme followed by extinction. Both groups of children acquired the instrumental response more slowly and exhibited more behavioral persistence (reduced extinction) when responding was initially reinforced under partial relative to continuous reinforcement, with no differences between groups. Progressive ratio stretching resulted in faster acquisition than partial reinforcement yet promoted equal behavioral persistence, again without differences between ADHD and TD groups. Unlike suggested by previous research, children with ADHD exhibit neither an acquisition deficit under partial reinforcement nor a deficit in PREE. Of relevance for Behavioral Treatment, gradual reinforcement stretching can be used to facilitate response acquisition over purely partial reinforcement while maintaining equal behavioral persistence upon reward discontinuation.

Evidence for motivational enhancement of sign-tracking behavior under reward uncertainty.

Reward uncertainty has been shown to invigorate rather than attenuate cue attraction and responding. For example, a number of findings have shown that partial reinforcement in autoshaping increases response rates to a conditioned stimulus (conditional stimulus) in comparison with continuous reinforcement. However, identifying the nature of this effect remains a topical question. The frustration theory posits that animals are frustrated by reward loss and predicts that enhanced responding results from higher response rates to conditional stimulus presentations that follow nonrewarded trials rather than rewarded trials. In contrast, the incentive hope hypothesis suggests that animals are motivated by possible future rewards and predicts similar response rates after rewarded and nonrewarded trials. Our results, which consist of a reanalysis of previously published data (Hellberg, Levit, & Robinson, 2018), are consistent with the incentive hope hypothesis because no differences were found between trials that follow rewarded or nonrewarded trials, or between trials that follow small or larger amounts of food reward in rats. There was also no evidence for an accumulation of frustration across each training session, with rats instead displaying enhanced yet stable responding from beginning to end. The incentive hope hypothesis is also briefly discussed in relation to the concept of incentive salience. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The partial-reinforcement extinction effect does not result from reduced sensitivity to nonreinforcement.

Five experiments used a magazine approach paradigm with rats to investigate whether learning about nonreinforcement is impaired in the presence of a conditioned stimulus (CS) that had been partially reinforced (PRf). Experiment 1 trained rats with a PRf CS and a continuously reinforced (CRf) CS, then extinguished responding to both CSs presented together as a compound. Probe trials of each CS presented alone revealed that extinction was slower for the PRf CS than the CRf CS, despite being extinguished in compound. In Experiment 2, a CRf light was extinguished in compound with either a CRf CS or a PRf CS that had been matched for overall reinforcement rate. Responding to the light extinguished at the same rate regardless of the reinforcement schedule of the other CS. Experiment 3 replicated this result with a PRf light. Thus, we found no evidence that a PRf CS impairs extinction of another CS presented at the same time. Experiments 4 and 5 extended this approach to study the acquisition of conditioned inhibition by training an inhibitor in compound with either a PRf or CRf excitatory CS. The reinforcement schedule of the excitatory CS had no effect on the acquisition of inhibition. In sum, conditioning with a PRf schedule slows subsequent extinction of that CS but does not affect learning about the nonreinforcement of other stimuli presented at the same time. We conclude that the Partial Reinforcement Extinction Effect is not attributable to a decrease in sensitivity to nonreinforcement following presentation of a PRf CS. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Safety Risk Analysis and Protective Control of Existing Pipelines Affected by Deep Pit Excavation in Metro Construction

Urban underground pipelines undertake many tasks closely related to people’s daily life and production, such as water supply, drainage, gas supply, and power supply. Metro projects are usually located in areas with dense underground pipelines; therefore, in the construction process of deep-foundation metro pit, some measures should be taken to protect the safety of underground pipelines. Under the engineering background of the Jiangbei foundation pit of the Wuhan Yangtze River Tunnel, the influence of soil disturbance caused by deep-foundation pit excavation is first simulated using the ABAQUS software on the displacement of underground pipelines and the dangerous area of underground pipelines is analyzed. Second, the most disadvantageous combination is analyzed for the risk factors of pipelines under the influences of multiple risk factors. Finally, a systematic additional partial reinforcement scheme (APRS) is proposed from the aspects of safety and economy to protect the safety of underground pipelines in the dangerous areas with greater risk. The results of measurement prove that the APRS is effective for pipeline protection and can provide reference for similar problems.

I feel safe when i know: Contingency instruction promotes threat extinction in high intolerance of uncertainty individuals

Extinction-resistant threat is considered to be a central feature of pathological anxiety. Reduced threat extinction is observed in individuals with high intolerance of uncertainty (IU). Here we sought to determine whether contingency instructions could alter the course of threat extinction for individuals high in IU. We tested this hypothesis in two identical experiments (Exp 1 n = 60, Exp 2 n = 82) where we recorded electrodermal activity during threat acquisition with partial reinforcement, and extinction. Participants were split into groups based on extinction instructions (instructed, uninstructed) and IU score (low, high). All groups displayed larger skin conductance responses to learned threat versus safety cues during threat acquisition, indicative of threat conditioning. In both experiments, only the uninstructed high IU groups displayed larger skin conductance responses to the learned threat versus safety cue during threat extinction. These findings suggest that uncertain threat during extinction maintains conditioned responding in individuals high in IU.

Aero-structural Investigation of Sesquiplane Wind Turbine Blades

Abstract The flexibility of large-scale wind turbines requires new structural blade designs to guarantee rigidity and avoid mechanical interference between the blades and tower. Segmented and partial reinforcement are effective approaches to improve the structural performance of large blades. This paper presents a novel sesquiplane blade design inspired by the biplane and sesquiplane aircraft, which have a similar development process. Wind tunnel testing and finite element simulation were conducted to investigate the aerodynamics and structural performance of the new design. The results indicate the sesquiplane blade, with airfoil sections in both the main and reinforcing spars, can notably enhance the integral rigidity and bearing capacity without a substantial loss in aerodynamic performance.

The partial reinforcement extinction effect: The proportion of trials reinforced during conditioning predicts the number of trials to extinction.

Four experiments compared the extinction of responding to a continuously reinforced (CRf) conditioned stimulus (conditional stimulus [CS]) consistently reinforced on every trial, with extinction of responding to a partially reinforced (PRf) CS that had been inconsistently reinforced. To equate the acquisition of responding between the two CSs, the average duration of the CRf CS was extended so that it scheduled the same overall rate of reinforcement per unit time as the PRf CS. Experiment 1 used a within-subjects design to compare the rates of extinction for a 10-s PRf CS reinforced on 33% of trials versus a 30-s CRf CS. Experiment 2 made the same comparison but using a between-subjects design. Experiment 3 compared extinction in a group trained with a 10-s PRf CS reinforced on 20% of trials and a group trained with a 50-s CRf CS. Experiment 4 compared the rates of extinction following two partial reinforcement schedules: a 10-s PRf CS reinforced on 33% of trial versus a 20-s CRf CS reinforced on 66% of trials. In each experiment, responding took longer to extinguish for the CS that scheduled a lower per-trial probability of reinforcement. Modeling of individual extinction curves using Weibull functions indicated that the latency to initiate extinction was directly related to the per-trial probability of reinforcement learned during acquisition. For example, compared with training with a CRf CS, rats reinforced on 33% of trials took approximately 3 times as many trials to initiate extinction, and rats reinforced on 20% of trials took 5 times as many trials to initiate extinction. These results provide support for trial-based accounts of extinction (e.g., Capaldi & Deutsch, 1967), whereby rats learn about the expected number of trials per reinforcer, and extinction depends on the number of expected reinforcers that have been omitted rather than on the number of extinction trials per se. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The re-artificial aging heat treatment by Laser for the purpose of hardness recovery to the partial reinforcement of piston due to the material substitution

The re-artificial aging heat treatment by Laser for the purpose of hardness recovery to the partial reinforcement of piston due to the material substitution

The Influence of Pain-Related Expectations on Intensity Perception of Nonpainful Somatosensory Stimuli.

The extent to which pain-related expectations, known to affect pain perception, also affect perception of nonpainful sensations remains unclear, as well as the potential role of unpredictability in this context. In a proprioceptive fear conditioning paradigm, various arm extension movements were associated with predictable and unpredictable electrocutaneous pain or its absence. During a subsequent test phase, nonpainful electrocutaneous stimuli with a high or low intensity were presented during movement execution. We used hierarchical drift diffusion modeling to examine the influence of expecting pain on the perceptual decision-making process underlying intensity perception of nonpainful sensations. In the first experiment (n = 36), the pain stimulus was never presented during the test phase after conditioning. In the second experiment (n = 39), partial reinforcement was adopted to prevent extinction of pain expectations. In both experiments, movements that were associated with (un)predictable pain led to higher pain expectancy, self-reported fear, unpleasantness, and arousal as compared with movements that were never paired with pain (effect sizes η2 ranging from 0.119 to 0.557; all p values < .05). Only in the second experiment-when the threat of pain remained present-we found that the expectation of pain affected decision making. Compared with the no pain condition, an a priori decision-making bias toward the high-intensity decision threshold was found with the strongest bias during unpredictable pain (effect sizes η2 ranging from 0.469 to 0.504; all p-values < .001). Thus, the expectation of pain affects inferential processes not only for subsequent painful but also for nonpainful bodily stimuli, with unpredictability moderating these effects, and only when the threat of pain remains present due to partial reinforcement.