IRT Distributions Research Articles

Network cache design under stationary requests: Exact analysis and Poisson approximation

The design of caching algorithms to maximize hit probability has been extensively studied. In this paper, we associate each content with a utility, which is a function of either the corresponding content hit rate. We formulate a cache optimization problem to maximize the sum of utilities over all contents under stationary and ergodic request processes. This problem is non-convex in general but we reformulate it as a convex optimization problem when the inter-request time (irt) distribution has a non-increasing hazard rate function. We provide explicit optimal solutions for some irt distributions. We formulate a reverse engineering based dual implementation of LRU under stationary arrivals. We also propose online algorithms that can be implemented using limited information and use a discrete time Lyapunov technique (DTLT) to correctly characterize their stability. Informed by these results, we further propose lightweight Poisson approximate online algorithms that are accurate and efficient in achieving optimal hit rates. We compare the solutions of the hit-rate based (HRB) and hit-probability based (HPB) problems. We find that online algorithms that solve HRB are more robust than online HPB algorithms.

Thermal image and spectral characterization of roadway failure process in geologically 45° inclined rocks

Large-scale geomechanical model test was conducted in order to investigate stability of an un-supported tunnel with rectangular cross section embedded in 45° inclined alternating strata of sandstone, mudstone and coal seam. The loading path consists of two groups: cases A–G with overburden depths from 296 to 948m and small loading rate, and cases H–M with overburden depths from 1126 to 2047m and fast loading rate. Infrared thermography, incorporated with image processing and Fourier transform, was employed to characterize the rock responses. Averaged temperature field, 〈IRT〉, represents energy release rate, oscillating at stick–slip pattern with different periods and amplitudes. Overburden depth and loading speed have a significant impact on 〈IRT〉 curve, i.e. small overburden depth and loading speed corresponds to long period and small amplitude; whereas, great overburden depth and fast loading speed to short period and high amplitude. The processed thermal image best represents rock behavior by two major IRT distribution modes. For loading cases A–G, the coal strata were over stressed indicated by high IRT while the mudstone strata were less stressed represented by low IRT, corresponding to the static interlayer friction. For loading cases H–M, the mudstone strata were over stressed indicated by high IRT while the coal strata were less stressed indicated by low IRT, corresponding to the dynamic friction. Fourier spectra and spatial frequency were employed to characterize the infrared sequence. Ultra-high spatial frequency component is a precursor for predicting the imminent dynamic event. Low spatial frequency component may be served as an indicator of the tunnel-wide sphere of influence that the stress redistribution extends.

A computational model of selection by consequences: Log survivor plots

[McDowell, J.J, 2004. A computational model of selection by consequences. J. Exp. Anal. Behav. 81, 297-317] instantiated the principle of selection by consequences in a virtual organism with an evolving repertoire of possible behaviors undergoing selection, reproduction, and mutation over many generations. The process is based on the computational approach, which is non-deterministic and rules-based. The model proposes a causal account for operant behavior. McDowell found that the virtual organism consistently showed a hyperbolic relationship between response and reinforcement rates according to the quantitative law of effect. To continue validation of the computational model, the present study examined its behavior on the molecular level by comparing the virtual organism's IRT distributions in the form of log survivor plots to findings from live organisms. Log survivor plots did not show the "broken-stick" feature indicative of distinct bouts and pauses in responding, although the bend in slope of the plots became more defined at low reinforcement rates. The shape of the virtual organism's log survivor plots was more consistent with the data on reinforced responding in pigeons. These results suggest that log survivor plot patterns of the virtual organism were generally consistent with the findings from live organisms providing further support for the computational model of selection by consequences as a viable account of operant behavior.

A Competition Model of IRT Distributions During the First Training Stages of Variable-Interval Schedule

A mathematical model based on the competition between neural networks is proposed. The model proposes that the neural network controlling the lever-pressing response during variable-interval schedules of reinforcement should compete with all other behavior-controlling networks. The network with the highest activity would be expressed as a behavior by a winner-take-all mechanism. This theoretical model fits the interresponse time distributions in 95% of cases at p<0.05 significance level, and 100% of cases at p<0.01, when tested with the Kolmogorov-Smirnoff test of goodness of fit. The model is discussed in psychological terms and possible generalizations are suggested.

Performance on a differential reinforcement of low-rate schedule in neodecorticated rats and rats with hippocampal lesions

Rats with total removal of neocortex (neodecorticates), hippocampectomized rats (hippocampals), and their controls were trained on a differential reinforcement of low-rate 12-sec (DRL12) schedule. With training, neodecorticates reduced the frequency of short interresponse times (ERTs) and increased the frequency of longer IRTs. They also significantly reduced the interresponse time per opportunity ratio for short IRTs and significantly increased the ratio in the period following reinforcement availability. Between-group differences on measures of IRT distribution, IRT per opportunity, and postreinforcement delay suggest separate contributions by neocortex and hippocampus to observed behavior on this task. Performance deficits in the lesioned groups are interpreted in terms of the associative power of the predictive lever increasing the pressure to respond in neodecorticates and the possible loss of place learning in both neodecorticates and hippocampals. In conjunction with previous work, these results indicate that temporal sensitivity in the rat is not destroyed by total neodecortication.

Accurate DRL performance in the pigeon: Comparison between perching and treadle pressing

Homing pigeons were reinforced for emitting a perching response according to differential-reinforcement-of-low-rate (DRL) schedules. The spacing requirement between successive perchings was progressively increased by 1-sec steps up to 70 sec and then abruptly decreased to 60, 40, and 20 sec. IRT/OP (interresponse time/opportunity) functions were maximal near the time of reinforcement. The coefficients of variation of the IRT distributions (ratio between the interquartile range and median IRT) fluctuated around .32, testifying for equivalent levels of adjustment throughout the critical IRT range. The ratio between reinforced and total IRTs ranged between .90 and .20. These data contrast with the performance of another group of pigeons reinforced for a treadle-pressing response according to DRL schedules (flatter IRT/OP functions, high coefficients of variation, and low efficiencies). Despite these differences in temporal regulation between perching and treadle-pressing DRL, response rates and reinforcement rates followed the same trend in both cases: they decreased as schedule value increased. The DRL perching results are similar to previous results obtained in the same species when perching duration was reinforced.

Retention and Relearning in Mult FR DRL and Mult FR DRO in Rats with Septal Lesions

The present experiment was designed to investigate the retention and relearning upon mult FR DRL and mult FR DRO schedule in rats with septal lesions.After thirteen days acquisition of mult FR10 DRL8, mult FR10 DRL20, mult FR10 DRO8, or mult FR10 DRO20 for food reinforcement, half of the rats received septal lesions and the other half received sham operations in each group. All of them were trained again on the same schedule as those of acquisition sessions for fourteen days after surgical operations. Using these procedures, every session was composed of the same three replications having a three min FR component, one min time out, and a seven min DRL or DRO component respectively. The reinforcer was a forty five mg pellet. The each experimental group was consisted of two animals in the relearning. In the case of septal lesions the electrode was lowered 2.0 mm anterior to bregma on the midline, 6.5mm below the surface of the skull, and 2. 0ma dc was passed for 30 secI. n the case of sham operations the electrode was lowered 2.0mm anterior to bregma on the midline, 3.5mm below the surface of the skull, no current was passed for 30 sec.The main findings were as follows. There were no significant differences in the number of lever press responses on FR component between septal and sham rats throughout the relearning. The efficiency ratio (reinforcements/responses) was lower in septal rats than in sham throughout the relearning (Fig. 2). The IRT distributions on DRL8 component showed that the responding of septalr ats as well as sham was suppressed by a particular class of IRTs.The septal area seems to be concerned with the withholding of operant responses on the DRL20 schedule which is more difficult to learn than the DRL8. The behavioral impairments of animals with septal lesions on DRL schedule may depend upon the DRL time. The septal area does not concern with the successive discrimination task which is required to differentiate the rate of operant responses for the corresponding component.

Effects of caffeine on DRL performance in the mouse

Mice were trained to stable and efficient DRL 18 sec performance utilizing a nose-poke as the operant. Caffeine, at doses less than 48 mg/kg, increased both response bursts (IRTs<3 sec) and longer IRTs, and shifted the IRT distribution towards shorter, nonreinforced IRTs. Auditory feedback for responses decreased the number of bursts emitted and produced performance more resistant to drug effects. These results are similar to those previously reported for caffeine on DRL in the rat, and for amphetamine on DRL in a variety of species.

Developmental Processes of Timing Behavior in mult FR DRL and mult FR DRO Schedules of Rats

Some developmental processes of timing behavior have been observed as the modification of a series of operant responses and some other collateral responses under several temporal reinforcement schedules, partic-ulary under DRL and DRO schedules (2, 7, 11, 12).The present study was designed to investigate such processes of timing behavior under the mult FR DRL and the mult FR DRO schedules, respectively. the Ss were 16 male rats of the in-bred Wister strain. They were experimentally naive and approximately 150 days old at the start of the experiment, weighing from 225g with a mean of 253g. After a feeding schedule of 7 days in which food intake was limited to maintain their body weights at about 85% of that in the ad-lib feeding condition, rats were adapted to a standard Skinner box having a response bar and a food tray, within a sound-attenuating room. The next day they were shaped to barpress for a 45mg food pellet for every bar pressing, and trained on a typical CRF schedule for the following 2 days. Thereafter, they were divided into two homogeneous groups of 8 rats. One group was trained with mult FR 5 DRL 8, and the other with mult FR 5 DRO 8. In these procedures, every session was composed of the same 3 replications having an FR component of 3 min, a TO of lmin, a DRL or DRO of 7 min, and a TO of 1 min, respectively. Therefore, every session time was 35min. The turning of a signal lamp and the presenting of a tone were used as the discriminative stimuli between the FR component and the DRL or the DRO component. The following day, each group was divided again into two homogeneous subgroups; the mult FR 5 DRL 8 group was divided into the mult FR 10 DRL 8 and the mult FR 10 DRL 20 groups, the mult FR 5 DRO 8 group into the mult FR 10 DRO 8 and the mult FR 10 DRO 20 groups. The training sessions for these four groups were continued once a day for 30 consecutive days.The main findings were as follows. The number of responses in each FR component, was gradually increased through the training sessions, but without significant group differences (Fig. 1). The distribution of pause time after the reinforced responses in FR 10 components did not show very many distinctive differences between groups or training sessions (Fig. 2). Fig. 3 indicates the number of responses in the DRL or the DRO components. The differences between groups were not so clear, although all groups decreased their responses with the lapse of the training course. The number of reinforced responses in the DRL or the DRO components was differed significantly in DRL or DRO time used, in the training sessions, and also in the reinforcement schedules applied. Furthermore, some interactions were found (Fig. 4). Fig. 5 and Fig. 6 show the IRT distributions obtained by the most typical individuals in the DRL and the DRO components. According to the training sessions carried out, the distributions of post SR intervals in DRO 8 and DRO 20 changed gradually to a more suitable direction for those reinforcement schedules (Fig. 7). The differences and the similarities of timing behavior between DRL and DRO schedules were discussed, and some advantages of experiments using the mult schedules to investigate timing behavior were suggested.

Effects of Septal Lesion on Operant Response and Collateral Behavior in DT-DRL of the Rat

The present study was designed to investigate the effect of septal lesion on operant response and collateral behavior in DT-DRL schedule of the rat. After the feeding schedule of 7 days in which food intake was limited to about 4% of ad lib body weight, 8 male rats of Wister strain were adapted to a Skinner box having a retractable (response) lever and a drinking spout for water intake. Next day, they were shaped to leverpress for a 45-mg food pellet every lever pressing. Then they were trained on a typical CRF schedule of 2 days. After then, the training procedure was changed to a DT-CRF of 5 days. In the first 2 days, DT time was started at 0.5 sec and gradually increased to 2 sec, and in the last 3 days DT time was fixed at 2 sec. According to the (operant) response results of each subjects in these stages, all rats were divided into 2 conditions; one of them was DT 2 DRL 6/LH 36 condition (N=4) and the other was DT-0 DRL 6/LH 36 condition (N=4). This stage was maintained for 3 consecutive days, and then only DRL time was increased to 20 seconds in each condition. The preoperative training procedure of DT-2 DRL 20/LH 36 or DT-0 DRL 20/LH 36 was carried out for 30 consecutive days. At the end of these training sessions, septal lesions were nitroduced. All surgery was performed under sodium pentobarbital anesthesia (Nembutal, 45mg/kg). In the half of subjects in each condition, septal lesions were made by passing 2.0 mA anodal d. c. for 30 sec, at the uninsulated tip of a stainless steel eelctrode inserted stereotaxically into the brain. An anal cathode completed the circuit. For septal lesions, the stereotaxical coordinates based on the rat brain atlas of PELLEGRINO & CUSHMAN were 2.0 mm anterior to bregma, and 5. 0 mm below the skull. Control lesions for remaining half of subjects in each condition, were made by only inserting the electrode to the corpus callosum at the same coordinates used for the septal lesions and withdrawing it for 30 sec without passing current. Following surgery, animals were allowed 3 days of recovery time with ad lib food and water intake, and the same feeding schedule used before surgery was applyed for 4 days. The postoperative retraining procedure was the same as the preoperative one, and the retraining sessions were carried out for 14 consecutive days. Following these behavioral testing, all subjects were sacrificed with overdose of anesthetic. The lesioned brain was perfused with 10 % formalin, and the block of a tissue containing the lesion was cut from the brain, dehydrated, embedded in paraffin, and sectioned at 20 p. Every fifth section was stained with hematoxylin-eosin for subsequent analysis of lesioned placement. The main behavioral findings were as follows. In the pre- and postoperative training sessions, the rats that belong to DT-2 condition showed more adaptive operant response than the rats in DT-0. After surgery, septal rats exhibited a great increase of lever pressings and poor reinforcements compared with those in the control rats or those of their own control in the last preoperative training session. But in the present experiment, it was very much important that the considerable recovery of DRL behavior was clear at the end of the retraining sessions, especially in the septal rats of DT-2 condition (Fig. 1 and Fig. 2). The analysis concerning to IRT distributions suggested that the effect of septal lesion in DT-2 was fairy different from that in DT-0. The mode of IRT distributions in DT-2 was slightly moved to sub-reinforcible IRT class just after the surgery, but in several days the mode of IRT distributions was returned to the level of the last preoperative session which was considered very excellent in DRL 20/LH 36 situation.

Effects of duration of DRL correlated stimuli on timing and collateral responding

Three pigeons were trained to respond under a two-key discrete trial multiple DRL4:DRL8 procedure in which the exteroceptive cues specifying the scheduled components were presented either throughout each trial or only at the start of each trial. Under continuous cue conditions, the IRT distributions for DRL 4 and DRL 8 components showed little overlap; modal IRT values for each component were slightly higher than the minimum reinforced value. Under brief cue conditions, all IRT distributions were similar, with modal values between 4 and 8 sec for both components. No clear relationship was found between noncontingent collateral responding and DRL value.

The effect of septal lesions in rats on performance of a free-operant avoidance task

Twenty female hooded rats of the Long-Evans strain served in a study of the effects of septal lesions on free-operant avoidance performance. Although there was no statistically significant difference between septal damaged and normal rats on responses emitted or shocks received, it was shown that normal animals emitted more responses than septal preparations in four second intervals following shock. Results are interpreted as demonstrating that shock is more disruptive to the bar-press response for septal damaged animals, and that this factor could partially account for the prevously reported different IRT distributions generated by septal damaged and normal animals on similar free operant avoidance tasks.

Contrasting effects of increased thirst and septal ablations on DRL responding in rats

Abstract The inter-response time distributions generated by 23 1 2 hr and 71 1 2 hr water deprived sham operated rats and 23 1 2 hr water deprived septal lesioned rats were compared on a DRL 22 sec schedule of water reinforcement. The results showed that septal ablations and increased thirst have markedly different effects on the IRT distributions generated by this schedule. These findings support the hypothesis that deficits in response inhibition associated with septal lesions result from the removal of the normal inhibitory influences mediated by this region.

THE EFFECT OF ACCELERATION ON FOOD‐REINFORCED DRL AND FR1

Performance on DRL 10 sec and FR 5 was studied after exposure to acceleration. After four rats, two on each of the above schedules, had stabilized they were exposed to 5 hr of acceleration at 5 G immediately before daily experimental sessions. Food intake was also studied in rats given access to food daily in their home cages and exposed to acceleration immediately before the free-feeding session. Weight gain of free-feeding animals and reinforcement intake of experimental animals dropped after acceleration. Over-all response rate on the FR was depressed markedly by acceleration but local response rates did not appear to be affected. IRT distributions of DRL sessions after acceleration were markedly shifted toward the long intervals. A sequential plot of IRTs on acceleration days showed an altered, but relatively stable, temporal patterning of responses followed by an abrupt return to the normal baseline toward the end of the session.

EFFECTS OF dl‐AMPHETAMINE UNDER CONCURRENT VI DRL REINFORCEMENT

Three adult, food-deprived rats were given IP injections of dl-amphetamine sulfate under DRL and concurrent VI DRL reinforcement schedules. The drug results were as follows.(1) The IRT distributions of DRL responses shifted to the left, but some temporal discrimination remained. (2) The IRT distributions of VI responses shifted slightly to the left. (3) The distinguishing characteristics of VI and DRL IRT distributions were preserved. (4) The frequency distribution of number of VI responses between two consecutive DRL responses was relatively unaffected. (5) Over-all response rates on the two components of the concurrent schedules increased more or less proportionately. These findings imply that the primary behavioral effect of dl-amphetamine was a motor excitatory one. The drug's disruption of timing behavior was not due to a derangement of internal timing mechanisms, nor to interference with the topography or pattern of behavior. Rather, it might be a secondary result of the accelerated emission of overt behavior patterns mediating the temporal spacing of DRL bar presses.