Successful Episodes Research Articles

Practice Reshapes the Geometry and Dynamics of Task-tailored Representations.

Extensive practice makes task performance more efficient and precise, leading to automaticity. However, theories of automaticity differ on which levels of task representations (e.g., low-level features, stimulus-response mappings, or high-level conjunctive memories of individual events) change with practice, despite predicting the same pattern of improvement (e.g., power law of practice). To resolve this controversy, we built on recent theoretical advances in understanding computations through neural population dynamics. Specifically, we hypothesized that practice optimizes the neural representational geometry of task representations to minimally separate the highest-level task contingencies needed for successful performance. This involves efficiently reaching conjunctive neural states that integrate task-critical features nonlinearly while abstracting over non-critical dimensions. To test this hypothesis, human participants (n = 40) engaged in extensive practice of a simple, context-dependent action selection task over 3 days while recording EEG. During initial rapid improvement in task performance, representations of the highest-level, context-specific conjunctions of task-features were enhanced as a function of the number of successful episodes. Crucially, only enhancement of these conjunctive representations, and not lower-order representations, predicted the power-law improvement in performance. Simultaneously, over sessions, these conjunctive neural states became more stable earlier in time and more aligned, abstracting over redundant task features, which correlated with offline performance gain in reducing switch costs. Thus, practice optimizes the dynamic representational geometry as task-tailored neural states that minimally tesselate the task space, taming their high-dimensionality.

Traffic navigation via reinforcement learning with episodic-guided prioritized experience replay

Deep Reinforcement Learning (DRL) models play a fundamental role in autonomous driving applications; however, they typically suffer from sample inefficiency because they often require many interactions with the environment to learn effective policies. This makes the training process time-consuming. To address this shortcoming, Prioritized Experience Replay (PER) has proven to be effective by prioritizing samples with high Temporal-Difference (TD) error for learning. In this context, this study contributes to artificial intelligence by proposing a sample-efficient DRL algorithm called Episodic-Guided Prioritized Experience Replay (EPER). The core innovation of EPER lies in the utilization of an episodic memory, dedicated to storing successful training episodes. Within this memory, expected returns for each state–action pair are extracted. These returns, combined with TD error-based prioritization, form a novel objective function for deep Q-network training. To prevent excessive determinism, EPER introduces exploration into the learning process by incorporating a regularization term into the objective function that allows exploration of state-space regions with diverse Q-values. The proposed EPER algorithm is suitable to train a DRL agent for handling episodic tasks, and it can be integrated into off-policy DRL models. EPER is employed for traffic navigation through scenarios such as highway driving, merging, roundabout, and intersection to showcase its application in engineering. The attained results denote that, compared with the PER and an additional state-of-the-art training technique, EPER is superior in expediting the training of the agent and learning a more optimal policy that leads to lower collision rates within the constructed navigation scenarios.

Mobile robot path planning using deep deterministic policy gradient with differential gaming (DDPG-DG) exploration

Mobile robot path planning involves decision-making in uncertain, dynamic conditions, where Reinforcement Learning (RL) algorithms excel in generating safe and optimal paths. The Deep Deterministic Policy Gradient (DDPG) is an RL technique focused on mobile robot navigation. RL algorithms must balance exploitation and exploration to enable effective learning. The balance between these actions directly impacts learning efficiency.This research proposes a method combining the DDPG strategy for exploitation with the Differential Gaming (DG) strategy for exploration. The DG algorithm ensures the mobile robot always reaches its target without collisions, thereby adding positive learning episodes to the memory buffer. An epsilon-greedy strategy determines whether to explore or exploit. When exploration is chosen, the DG algorithm is employed. The combination of DG strategy with DDPG facilitates faster learning by increasing the number of successful episodes and reducing the number of failure episodes in the experience buffer. The DDPG algorithm supports continuous state and action spaces, resulting in smoother, non-jerky movements and improved control over the turns when navigating obstacles. Reward shaping considers finer details, ensuring even small advantages in each iteration contribute to learning.Through diverse test scenarios, it is demonstrated that DG exploration, compared to random exploration, results in an average increase of 389% in successful target reaches and a 39% decrease in collisions. Additionally, DG exploration shows a 69% improvement in the number of episodes where convergence is achieved within a maximum of 2000 steps.

ESTABLISHED PRACTICES, CHALLENGES AND ACHIEVEMENTS IN CREATING INCLUSIVE ENGLISH LANGUAGE CLASSROOMS IN OMAN’S INSTITUTIONS OF HIGHER EDUCATION

Research in the context of Oman’s higher education indicates that well-established support service systems and procedures, innovations in technology and English language pedagogy could be crucial for special needs undergraduate students’ language learning and their effective inclusion into the classroom. However, limited attention has still been reported on the common teaching and learning practices, as well as challenging and successful episodes of such practices that could be incremental to the context. This design-based study yielded data and descriptive information collected in order to better understand teachers’ unique perspectives and opinions about English language inclusive teaching practices currently employed across higher education institutions in Oman. Accounts on challenges and achievements in creating inclusive English language classroom were gathered through the structured interviews with six faculty of the Centre for Preparatory Studies at Sultan Qaboos University. The analysis involved data management, description, and identification of the themes that brought meaning and unified diverse experiences into meaningful wholes. The results of the study reveal sensitivity and very positive attitude shown by teachers toward inclusive English language teaching. However, there are several challenging aspects, including misinterpreted concepts, limited knowledge and expertise. Though the study looked only at teachers from one institutions of higher education, it is evident that search for both formal and informal means of helping teachers in developing new insights into inclusive language pedagogy is essential. Being supplied with access to relevant teacher professional development and resources on innovations in education and technology, the teaching community will be provided with better pedagogical tools, and many barriers in teaching and learning will be removed.

Learning by Successful or Unsuccessful Experiences?

Humans learn from both successful and unsuccessful experiences, because useful information about how to solve complex problems can be gleaned not only from success but also from failure. In this paper, we propose a method for investigating this difference by applying Preference based Inverse Reinforcement Learning to Double Transition Models built from replays of StarCraft II. Our method provides two advantages: (1) the ability to identify integrated reward distributions from computational models composed of multiple experiences, and (2) the ability to discern differences between learning by successes and failures. Our experimental results demonstrate that reward distributions are shaped depending on the trajectories utilized to build models. Reward distributions based on successful episodes were skewed to the left, while those based on unsuccessful episodes were skewed to the right. Furthermore, we found that players with symmetric triple reward distributions had a high probability of winning the game.

How Exporters Grow

Abstract We use customs data for Irish firms to show that in successful episodes of export market entry, there are statistically and economically significant post-entry dynamics of quantities, but not of mark-ups. To match these moments, we structurally estimate a model where firms can invest in future customer base through two channels: by selling more today, and by spending on marketing and advertising. Our estimates suggest that customer base is insensitive to lagged sales, so firms have no incentive to engage in dynamic pricing to accumulate customers. Instead, investment in customer base through marketing and advertising explains the dynamics of quantities. The ratio of advertising and marketing expenditures to sales implied by the model is consistent with data from other sources. Our estimated model generates long-run export responses to permanent tariff changes that are bigger than short run responses, as well as responses that are increasing in the expected persistence of tariff shocks, contributing to our understanding of the International Elasticity Puzzle.

Горское движение и белая армия на Северном Кавказе в 1918-1919 гг.: история несостояв-шегося союза

The article is devoted to the problems related to the possible military-political alliance of the White Guards of the south of Russia and the mountain liberal figures during 1918-1919. It is proved that the prospects for such a combination of efforts were fully justified. Throughout 1917, the mountain and Terek Cossack leaders established quite successful relations based on a general rejection of the growing influence of the Bolsheviks in the Caucasus region. Attention is focused on the successful episodes of the anti-Bolshevik struggle: the Provisional Terek-Dagestan gov-ernment, the Terek anti-Bolshevik uprising of 1918, the armed struggle against the Bolsheviks in the North Caucasus. Conclusions are drawn that despite the obvious mutual benefit from joining forces to fight the Bolsheviks, the political ambitions of the leadership of the White Guards and mountain activists prevailed and eventually led to the defeat of all anti-Bolshevik forces in the North Caucasus.

JK07 prevents atrial fibrosis and atrial fibrillation inducibility in a porcine DOCA model

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Geconcerteerde Onderzoeksactie (GOA) grant 499 of the University of Antwerp Industrieel OnderzoeksFonds/Strategisch Basisonderzoek 498 (IOF/SBO) research grant of the University of Antwerp Background Atrial fibrosis is a substrate of atrial fibrillation (AF), and higher burdens of fibrosis are associated with resistance to therapy and worse prognosis. Currently, no therapies exist that target atrial fibrosis. JK07 is a long-acting neuregulin-fusion protein that has been shown to decrease ventricular fibrosis in models of heart failure through selective stimulation of the ErbB4 receptor. Purpose To test the ability of JK07 to reduce atrial fibrosis and AF inducibility in minipig model of deoxycorticosterone acetate (DOCA, an aldosterone agonist) induced hypertension. Methods 18 Aachener minipigs were randomized into 3 groups: control (CTRL), DOCA + Vehicle (DOCA+VEH) and DOCA+JK07. The control group did not undergo a therapeutic intervention. To induce hypertension and atrial fibrosis, pellets releasing 10 mg/kg DOCA over a period of 60 days were implanted in minipigs in the DOCA+VEH and DOCA+JK07 groups. The DOCA-implanted animals underwent weekly treatment with JK07 (0.3 mg/kg; DOCA + JK07) or its vehicle (DOCA + VEH), starting at the day of implantation (total of 9 administrations). After 60 days, arterial blood pressure was measured invasively and a decapolar catheter was placed in the right atrium. AF inducibility was tested by performing 50 burst pacing episodes and quantified as the percentage of successful episodes where an AF run ≥ 5 sec could be induced after the burst, out of the 50 attempts. Atrial fibrosis was quantified using ImageJ software on Masson trichrome staining of left atrial specimens, isolated after euthanizing the animals. Results Mean arterial pressure was significantly higher in the DOCA+VEH (142 ± 10 mmHg) and DOCA+JK07 (132 ± 15 mmHg) groups than in the CTRL group (105 ± 8 mmHg, p<0.01), without an effect of JK07. AF inducibility in the DOCA+VEH group was significantly higher than in the CTRL group (66/250 vs. 9/300, p<0.001), and the DOCA + JK07 group (66/250 vs. 8/300, p<0.001). Likewise, the degree of atrial fibrosis was significantly higher in the DOCA+VEH group compared to CTRL (14.18 ± 1.81 vs. 8.30 ± 2.52, p<0.001) and compared to DOCA+JK07 (14.18 ± 1.81 vs. 10.65 ± 1.59, p=0.0049). Conclusions JK07 prevents atrial fibrosis and AF inducibility in a porcine DOCA model. The effect of JK07 is unrelated to effects on blood pressure, but probably related to reduced atrial fibrogenesis.

Identifying an optimal ejaculation latency for the diagnosis of men reporting orgasmic/ejaculation difficulty.

Criteria for the definition and diagnosis of delayed ejaculation (DE) are yet under consideration. This study sought to determine an optimal ejaculation latency (EL) threshold for the diagnosis of men with DE by exploring the relationship between various ELs and independent characterizations of delayed ejaculation. In a multinational survey, 1660 men, with and without concomitant erectile dysfunction (ED) and meeting inclusion criteria, provided information on their estimated EL, measures of DE symptomology, and other covariates known to be associated with DE. We determined an optimal diagnostic EL threshold for men with DE. The strongest relationship between EL and orgasmic difficulty occurred when the latter was defined by a combination of items related to difficulty reaching orgasm and percent of successful episodes in reaching orgasm during partnered sex. An EL of ≥16minutes provided the greatest balance between measures of sensitivity and specificity; a latency ≥11minutes was the best threshold for tagging the highest number/percentage of men with the severest level of orgasmic difficulty, but this threshold also demonstrated lower specificity. These patterns persisted even when explanatory covariates known to affect orgasmic function/dysfunction were included in a multivariate model. Differences between samples of men with and without concomitant ED were negligible. In addition to assessing a man's difficulty reaching orgasm/ejaculation during partnered sex and the percent of episodes reaching orgasm, an algorithm for the diagnosis of DE should consider an EL threshold in order to control diagnostic errors. This study is the first to specify an empirically supported procedure for diagnosing DE. Cautions include the use of social media for participant recruitment, relying on estimated rather than clocked EL, not testing for differences between DE men with lifelong vs acquired etiologies, and the lower specificity associated with using the 11-minute criterion that could increase the probability of including false positives. In diagnosing men with DE, after establishing a man's difficulty reaching orgasm/ejaculation during partnered sex, using an EL of 10 to 11minutes will help control type 2 (false negative) diagnostic errors when used in conjunction with other diagnostic criteria. Whether or not the man has concomitant ED does not appear to affect the utility of this procedure.

Imaginary filtered hindsight experience replay for UAV tracking dynamic targets in large-scale unknown environments

As an advanced combat weapon, Unmanned Aerial Vehicles (UAVs) have been widely used in military wars. In this paper, we formulated the Autonomous Navigation Control (ANC) problem of UAVs as a Markov Decision Process (MDP) and proposed a novel Deep Reinforcement Learning (DRL) method to allow UAVs to perform dynamic target tracking tasks in large-scale unknown environments. To solve the problem of limited training experience, the proposed Imaginary Filtered Hindsight Experience Replay (IFHER) generates successful episodes by reasonably imagining the target trajectory in the failed episode to augment the experiences. The well-designed goal, episode, and quality filtering strategies ensure that only high-quality augmented experiences can be stored, while the sampling filtering strategy of IFHER ensures that these stored augmented experiences can be fully learned according to their high priorities. By training in a complex environment constructed based on the parameters of a real UAV, the proposed IFHER algorithm improves the convergence speed by 28.99% and the convergence result by 11.57% compared to the state-of-the-art Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The testing experiments carried out in environments with different complexities demonstrate the strong robustness and generalization ability of the IFHER agent. Moreover, the flight trajectory of the IFHER agent shows the superiority of the learned policy and the practical application value of the algorithm.

An analysis of the success of fiscal adjustment in reducing public debt: Evidence from Pakistan.

This study seeks answers to questions such as: what is fiscal adjustment? Which fiscal strategy will result in a reduction in public debt liabilities? In the pursuit of answers to these questions, the study has defined two objectives. Firstly, the fiscal adjustment episodes must be identified in order to detach the discretionary fiscal stance; secondly, the success of these adjustment episodes in reducing public debt liabilities must be assessed. As a result, attempts will be made to undertake analyses that would simplify the issues underlying Pakistan’s practical policy options. A total of eleven adjustment episodes have been observed in a sample, ranging from 1976 to 2017, following Alesina and Ardagna’s definition. The descriptive analysis reveals that five episodes succeeded in reducing the public debt, while six episodes failed to reduce the ratio. Out of the five successful episodes, four are found to be spending-based and one is tax-based. To quantify the success of fiscal adjustment, the empirical model has been calibrated on Leibrecht and Scharler’s model and estimation is done via both the Ordinary Least Squares (OLS) and Robust Least Squares (RLS) methods. The RLS method produces better outcomes than the OLS method. Under RLS, all variables are significant except GDP growth, whereas in the OLS model, the election year and regime shift, together with GDP growth, are statistically insignificant. The fiscal adjustment’s composition reveals that spending-based consolidation boosts the chances of the fiscal adjustment’s success. Fiscal authorities should, therefore, adopt spending-based austerity measures to ensure the sustainability of public finances and prevent the negative macroeconomic consequences of unsustainable public debt.

The human cognitive system corrects traces of error commission on the fly.

Human perception and action rely on a fundamental binding mechanism that forges integrated event representations from distributed features. Encountering any one of these features later on can retrieve the whole event, thus expediting cognitive processing. The traditional view on binding confines it to successful action episodes, holding that the human cognitive system does not leverage errors for optimizing corresponding event representations. Here we use sequential analyses of erroneous action episodes to explore whether binding promotes future successful behavior even when actions go awry. Results indicate that the processes leading to binding integrate different aspects of the action episode in a highly efficient and flexible manner to privilege future correct actions and prepare the ground for error-based learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The role of gaze in meaning negotiation episodes in video synchronous computer-mediated interactions

Abstract The outbreak of COVID-19 led to the replacement of many traditional classroom teaching and learning activities with online interactions through synchronous audio or video conferencing tools. Technically, the key difference between audio and video conferencing is the latter’s use of a webcam. Therefore, this study aims to investigate the webcam’s role in video synchronous computer-mediated communication (SCMC). Negotiation for meaning episodes, in which interlocutors try to resolve non-understanding through verbal interaction, were selected for analysis because they are widely believed to be helpful for second language acquisition. Specifically, this paper focuses on where interlocutors look during negotiated interactions in video SCMC and whether a statistical relationship exists between looking at a peer’s video image and successful meaning negotiation. Four dyads of English learners completed information gap tasks with potentially unknown words through learner–learner video SCMS interactions. Students’ multimodal performance was screen recorded for a gaze analysis and one-on-one interviews were conducted to confirm the students’ gaze directions. The results reveal a statistically significant positive relationship between the duration of one’s gaze at the peer’s video image and the number of successful meaning negotiation episodes in a dyad.

Solving hard-exploration problems with counting and replay approach

The reinforcement learning agent has been very successful in many Atari 2600 games. However, while applied to a more complex and challenging environment, it is crucial to avoid falling into the local optimum, especially when the games contain many traps, ample action space, challenging scenarios, and sporadic successful episodes. In this case, using the intrinsic motivation method can easily fall into the local optimum. If the domain knowledge is excessively used, it is not applicable when encountering different game designs. Therefore, to enhance the agent’s ability to explore and avoid catastrophic forgetting due to the fades of intrinsic motivation, a Trajectory Evaluation Module is developed and integrated with ideas from the Count-Based Exploration and Trajectory Replay method.Moreover, our approach is integrated very well with the Self Imitation Learning method and works effectively for hard-exploration video games. Our policy is also evaluated with two video games: Super Mario Bros and Sonic the Hedgehog. The experiment results show that our Trajectory Evaluation Module can help the agent pass through various obstacles and scenarios, and successfully break through all levels of Super Mario Bros.

Deepening Democracy? Promises and challenges of Chile’s Road to a New Constitution

As a response to mass mobilizations against the political and social status quo, a multiparty agreement activated a process to replace the constitution in Chile, three decades after the country’s transition to democracy. I argue that this process has three features that are not only desirable on normative grounds but also shared by successful episodes of constitutional replacement in democratic regimes: the drafting of the new text in an assembly bound by preexisting rules, inclusive mechanisms of representation and decision making, and direct citizen involvement. These features, in interaction with the popular repudiation of the elite-biased institutions inherited from the Pinochet era, might lead to a new constitutional arrangement that deepens democratization by expanding citizen rights, strengthening executive constraints, and removing restrictions on majority rule. Yet the realization of the full democratic potential of this constitutional change depends on the still uncertain ability of constitution makers to compromise on a text that is not only seen as legitimate among ordinary citizens but also promotes coherent goals, effective institutions, and good governance.

Plasmodium vivax pre-erythrocytic vaccines.

An estimated 229 million cases of malaria occurred worldwide in 2019. Both, Plasmodium falciparum and P. vivax are responsible for most of the malaria disease burden in the world. Despite difficulties in obtaining an accurate number, the global estimates of cases in 2019 are approximately 229 million of which 2.8% are due to P. vivax, and the total number of malaria deaths are approximately 409 million. Regional elimination or global eradication of malaria will be a difficult task, particularly for P. vivax due to the particular biological features related to the hypnozoite, leading to relapse. Countries that have shown successful episodes of a decrease in P. falciparum malaria, are left with remaining P. vivax malaria cases. This is caused by the mechanism that the parasite has evolved to remain dormant in the liver forming hypnozoites. Furthermore, while clinical trials of vaccines against P. falciparum are making fast progress, a very different picture is seen with P. vivax, where only few candidates are currently active in clinical trials. We discuss the challenge that represent the hypnozoite for P. vivax vaccine development, the potential of Controlled Human Malaria Challenges (CHMI) and the leading vaccine candidates assessed in clinical trials.

Accelerating Economic Growth: The Science beneath the Art

Rapid and sustained accelerations in economic growth can have huge implications for poverty alleviation and people's wellbeing, but does the economic profession have the knowledge to engineer them? Conspicuously disappointing outcomes for countries scrupulously following mainstream growth policy recipes have sown some doubt. This paper deploys a novel statistical approach to show that 80% of 135 successful growth acceleration episodes – occurring between 1962 and 2002 worldwide – were preceded by major improvements in standard growth determinants, and particularly orthodox policies such as economic liberalisations. However, a set of counterfactual analyses suggest that such major improvements in growth determinants are a necessary but hardly sufficient condition for success, failing to accelerate growth in 9 out of 10 instances. Our overall results indicate that to increase their effectiveness, acceleration strategies should remain focussed on mainstream growth recipes, but abandon off-the-shelf approaches or all-encompassing “shock therapy” solutions, favouring improved tailoring to local economic conditions.

Vain or able? Strong inference and the efficacy debate

Purpose This study aims to address the efficacy debate by exploring the nature of how prior team level performance affects future performance. That is, the purpose of this study is to understand whether or not the boost of efficacy associated with success leads to overconfidence that harms performance or to motivation that enhances performance. Design/methodology/approach This study used a quantitative approach to test competing hypotheses derived from both social cognitive theory and control theory. Specifically, the study made use of archival National Football League data, containing 5,120 longitudinal team level observations. This paper uses multi-level modeling to analyze how prior team level performance affected future performance episodes. Findings The findings of this study suggest that prior success leads to overconfidence which ultimately harms future team performance. Therefore, the findings support control theory in favor of the social cognitive theory. However, this study finds that the detrimental effects of overconfidence could be offset by monitoring and work breaks. Research limitations/implications Due to the nature of the archival data source, it was not possible to directly measure efficacy. Thus, efficacy is inferred based on past performance outcomes. Practical implications This study suggests that it is important for managers and team leaders to pay careful attention to their team after successful performances. Specifically, team leaders may want to monitor their members or give them a break after successful performance episodes to avoid the negative effects of overconfidence. Originality/value This paper provides a direct test of the efficacy debate at the team level.

Successful and Failed Episodes of Democratization: Conceptualization, Identification, and Description

Successful and Failed Episodes of Democratization: Conceptualization, Identification, and Description

LABOR ZIONIST IDEOLOGY AND THE FOUNDATION OF ISRAELI FOREIGN POLICY

The paper argues that during the Mandate period the Labor Zionist movement was able to successfully create a Sabra identity based on its ideology that was constructed in opposition to the presumed characteristics of the “exile Jew” and how such an identity played a central role in the formation of a security oriented foreign policy. Labor's creation of the Sabra through the "Hebrew Revolution" can be considered as one of the most successful episodes of the twentieth century in which a new identity was created in order to serve ideological goals. Labor's Zionist ideology, which sought to create a “new Jew” that would form the basis of the Jewish national movement, was translated into an identity that in contrast to the diaspora Jew relied on collectivism, agriculture, secularism, and most important of all physical strength and sacrifice in defence of the Jewish nation. This translated into a security-oriented foreign policy that heavily relied on military force and emphasized internal power and strength, which Labor elites argued could only be achieved through self-reliance and independence particularly in regards to defence issues. Such an orientation would form the basis of Israeli foreign policy for years to come.