Exploratory Actions Research Articles

Goal Recognition with Markov Logic Networks for Player-Adaptive Games

Goal recognition in digital games involves inferring players’ goals from observed sequences of low-level player actions. Goal recognition models support player-adaptive digital games, which dynamically augment game events in response to player choices for a range of applications, including entertainment, training, and education. However, digital games pose significant challenges for goal recognition, such as exploratory actions and ill-defined goals. This paper presents a goal recognition framework based on Markov logic networks (MLNs). The model’s parameters are directly learned from a corpus that was collected from player interactions with a non-linear educational game. An empirical evaluation demonstrates that the MLN goal recognition framework accurately predicts players’ goals in a game environment with exploratory actions and ill-defined goals.

How tufted capuchin monkeys (Sapajus spp.) and humans (Homo sapiens) handle a jointed tool.

The embodied theory of tooling predicts that when using a grasped object as a tool, individuals accommodate their actions to manage the altered degrees of freedom in the body-plus-object system. We tested predictions from this theory by studying how 3 tufted capuchin monkeys (Sapajus spp.) and 6 humans (Homo sapiens) used a hoe to retrieve a token. The hoe's handle was rigid, had 2 segments with 1 planar joint, or had 3 segments with 2 (orthogonal) planar joints. When jointed, rotating the handle could render it rigid. The monkeys used more actions to retrieve the token when the handle had 1 joint than when it had no joints or 2 joints. They did not use exploratory actions frequently nor in a directed manner in any condition. Although they sometimes rotated the handle of the hoe, they did not make it rigid. In a follow-up study, we explored whether humans would rotate the handle to use a 2-jointed hoe in a conventional manner, as predicted both by the embodied theory and theories of functional fixedness in humans. Two people rotated the handle to use the hoe conventionally, but 4 people did not; instead, they used the hoe as it was presented, as did the monkeys. These results confirm some predictions but also highlight shortcomings of the embodied theory with respect to specifying the consequences of adding multiple degrees of freedom. The study of species' perceptual sensitivity to jointed object's inertial properties could help to refine the embodied theory of tooling. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Optimal adaptive inspection and maintenance for redundant systems

Optimal exploration of engineering systems can be guided by the principle of Value of Information (VoI), which accounts for the topological important of components, their reliability and the management costs. For series systems, in most cases higher inspection priority should be given to unreliable components. For redundant systems such as parallel systems, analysis of one-shot decision problems shows that higher inspection priority should be given to more reliable components. This paper investigates the optimal exploration of redundant systems in long-term decision making with sequential inspection and repairing. When the expected, cumulated, discounted cost is considered, it may become more efficient to give higher inspection priority to less reliable components, in order to preserve system redundancy. To investigate this problem, we develop a Partially Observable Markov Decision Process (POMDP) framework for sequential inspection and maintenance of redundant systems, where the VoI analysis is embedded in the optimal selection of exploratory actions. We investigate the use of alternative approximate POMDP solvers for parallel and more general systems, compare their computation complexities and performance, and show how the inspection priorities depend on the economic discount factor, the degradation rate, the inspection precision, and the repair cost.

First evidence of stone handling in geladas: From simple to more complex forms of object play

Stone Handling (SH) is a solitary object play behaviour that can vary from simple exploratory actions to more complex manipulations. So far, among primates, this behaviour has been exclusively reported in macaques. We collected data on 62 geladas (Theropithecus gelada) housed at the NaturZoo (Rheine, Germany). We found that about 70% of subjects belonging to all age- and sex-classes engaged in SH. Due to their exceptional manipulative skills (the highest opposability index among nonhuman primates) and propensity to play, geladas are a good model species to test hypotheses on the function of this form of object play. While the frequency of SH tended to decrease with age of the player, the duration of each session and its complexity tended to increase in juveniles/subadults and adults compared to infants. This age-related variation in terms of frequency, duration and complexity suggests that, in agreement with the motor training hypothesis, SH could have a role in the neural–motor development of immature subjects and a basic function in stimulating neurogenesis and maintaining the psychological well-being of adults. In all age classes, the frequency of SH did not vary across pre-feeding, feeding and non-feeding conditions. Hence, our data do not support the misdirected foraging hypothesis, which predicts that animals engage in SH to anticipate food provisioning. In conclusion, our study reveals, for the first time, the presence of SH outside the genus Macaca and attempts to delineate possible functions of the behaviour in geladas. Since the hypotheses tested cannot be mutually exclusive, long-term studies of SH across individuals’ lifetimes in both captive and wild groups of geladas are needed to clarify the proximate and ultimate functions of the behaviour in this species of Papionini. Finally, long-term studies could also provide some important indications about the cultural nature and social transmission of SH in a taxonomic group outside the Macaca genus.

Adult and juvenile bearded capuchin monkeys handle stone hammers differently during nut‐cracking

Wild bearded capuchin monkeys (Sapajus libidinosus) habitually use stone hammers to crack open palm nuts and seeds on anvils. This activity requires strength, balance, and precise movement of a large stone with respect to the item placed on an anvil. We explored how well young monkeys cope with these challenges by examining their behavior and the behavior of adults while they cracked palm nuts using a stone. Using video records, we compared actions of six juvenile (2-5 years) and six adult (7+ years) wild monkeys during their first 20 strikes with one unfamiliar ellipsoid, quartzite stone (540 g), and the outcomes of these strikes. Compared with adults, juveniles cracked fewer nuts, performed a more diverse set of exploratory actions, and less frequently placed one or both hands on top of the stone on the downward motion. Adults and juveniles displayed similar low frequencies of striking with a slanted trajectory, missing the nut, and losing control over the nut or stone after striking. These findings indicate that young monkeys control the trajectory of a stone adequately but that is not sufficient to crack nuts as effectively as adults do. Compared with juveniles, adults more quickly perceive how to grip the stone efficiently, and they are able to adjust their grip dynamically during the strike. Young monkeys develop expertise in the latter aspects of cracking nuts over the course of several years of regular practice, indicating that perceptual learning about these aspects of percussion occurs slowly. Juvenile and adult humans learning to use stones to crack nuts also master these features of cracking nuts very slowly.

The association between visual exploration and passing performance in high-level U13 and U23 football players

ABSTRACT Purpose: The visual exploratory actions (i.e., scanning head movements) used by football players to perceive their surrounding environment have recently gained interest. While this has resulted in important findings relating to visual exploration during natural match-play, often the study designs lacked the experimental control of laboratory-based experimental settings. We aimed to investigate whether visual exploratory action is associated with passing performance for high-level U13 and U23 players in a controlled skill assessment setting. Methods: Fourteen U13 and 13 U23 football players from a Bundesliga club completed a standardised 32-trial sequence in the Footbonaut. Exploratory head movements were recorded with a head-worn inertial sensor, from which the count, frequency and excursion of head movements were extracted before and during ball possession. Ball reception and disposal were coded for each trial, and performance was operationalised as the time taken to complete each trial. Results: Across all players, visual exploratory action was associated with passing performance. The variables that best explained faster performance were 1) a higher number of head turns before receiving the ball, 2) a lower number of head turns when in possession of the ball, and 3) being an U23 player. However, different combinations of variables explained performance for U13 and U23 players. Conclusion: The findings demonstrate the value of scanning before receiving the ball to prospectively control passing actions in the Footbonaut. Future research should investigate the shared and contrasting characteristics of scanning actions, as they are observed by players in skill assessment tasks such as the Footbonaut, during training and during match-play.

Child-Robot Collaborative Problem-Solving and the Importance of Child's Voluntary Interaction: A Developmental Perspective.

The emergence and development of cognitive strategies for the transition from exploratory actions towards intentional problem-solving in children is a key question for the understanding of the development of human cognition. Researchers in developmental psychology have studied cognitive strategies and have highlighted the catalytic role of the social environment. However, it is not yet adequately understood how this capacity emerges and develops in biological systems when they perform a problem-solving task in collaboration with a robotic social agent. This paper presents an empirical study in a human-robot interaction (HRI) setting which investigates children's problem-solving from a developmental perspective. In order to theoretically conceptualize children's developmental process of problem-solving in HRI context, we use principles based on the intuitive theory and we take into consideration existing research on executive functions with a focus on inhibitory control. We considered the paradigm of the Tower of Hanoi and we conducted an HRI behavioral experiment to evaluate task performance. We designed two types of robot interventions, “voluntary” and “turn-taking”—manipulating exclusively the timing of the intervention. Our results indicate that the children who participated in the voluntary interaction setting showed a better performance in the problem solving activity during the evaluation session despite their large variability in the frequency of self-initiated interactions with the robot. Additionally, we present a detailed description of the problem-solving trajectory for a representative single case-study, which reveals specific developmental patterns in the context of the specific task. Implications and future work are discussed regarding the development of intelligent robotic systems that allow child-initiated interaction as well as targeted and not constant robot interventions.

Land-use and coverage in the structure of the landscape in a Tropical Dry Forest in northeast Brazil

Landscape Ecology using GIS is applied to understand the effects of forest fragmentation and makes it feasible mainly for extensive areas, through the possibility of analyzing both the variety of units that compose it in time. This study aims to analyze the land use and coverage of tropical dry forests and to characterize their landscape structure in an area of the municipality of Floresta, Pernambuco, Brazil, in order to understand how the landscape configuration can influence the conservation. Orbital images were used from LANDSAT, and the classification was performed in the QGIS 2.18 software using the Maximum Likelihood methodification, for the years of 2007 and 2017. A circular buffer was incorporated around the area of study to analyze the structural patterns in the context of the landscape. Five thematic classes were used: dense vegetation, sparse vegetation, exposed soil, grassland, and water. The metrics were obtained through the Patch Analyst tool only to the class of dense vegetation. It is noted that much of the dense vegetation has become sparse. There was a reduction of exposed soil and an increase in grassland areas, which may indicate a regeneration process of the vegetation. The fragments of dense vegetation with more substantial dimensions are on constant exploratory actions and are the most affected with the reduction of size. Furthermore, climatic factors, periods of drought, and the presence of livestock may also affect the fragmentation of dense vegetation patches. Therefore, the landscape became heterogeneous, and a tendency was observed to increase the fragmentation, mainly in the more significant spots.

Constraints on visual exploration of youth football players during 11v11 match-play: The influence of playing role, pitch position and phase of play

ABSTRACT Visual exploratory action, in which football players turn their head to perceive their environment, improves prospective performance with the ball during match-play. This scanning action, however, is relevant for players throughout the entire match, as the information perceived through visual exploration is needed to guide movement around the pitch during both offensive and defensive play. This study aimed to understand how a player’s on-pitch position, playing role and phase of play influenced the visual exploratory head movements of players during 11v11 match-play. Twenty-two competitive-elite youth footballers (M = 16.25 years) played a total of 1,623 minutes (M = 73.8). Inertial measurement units, global positioning system units and notational analysis were used to quantify relevant variables. Analyses revealed that players explored more extensively when they were in possession of the ball, and less extensively during transition phases, as compared to team ball-possession and opposition ball-possession phases of play. Players explored most extensively when in the back third of the pitch, and least when they were in the middle third of the pitch. Playing role, pitch position and phase of play should be considered as constraints on visual exploratory actions when developing training situations aimed at improving the scanning actions of players.

Object shape estimation and modeling, based on sparse Gaussian process implicit surfaces, combining visual data and tactile exploration

Inferring and representing three-dimensional shapes is an important part of robotic perception. However, it is challenging to build accurate models of novel objects based on real sensory data, because observed data is typically incomplete and noisy. Furthermore, imperfect sensory data suggests that uncertainty about shapes should be explicitly modeled during shape estimation. Such uncertainty models can usefully enable exploratory action planning for maximum information gain and efficient use of data. This paper presents a probabilistic approach for acquiring object models, based on visual and tactile data. We study Gaussian Process Implicit Surface (GPIS) representation. GPIS enables a non-parametric probabilistic reconstruction of object surfaces from 3D data points, while also providing a principled approach to encode the uncertainty associated with each region of the reconstruction. We investigate different configurations for GPIS, and interpret an object surface as the level-set of an underlying sparse GP. Experiments are performed on both synthetic data, and also real data sets obtained from two different robots physically interacting with objects. We evaluate performance by assessing how close the reconstructed surfaces are to ground-truth object models. We also evaluate how well objects from different categories are clustered, based on the reconstructed surface shapes. Results show that sparse GPs enable a reliable approximation to the full GP solution, and the proposed method yields adequate surface representations to distinguish objects. Additionally the presented approach is shown to provide computational efficiency, and also efficient use of the robot’s exploratory actions.

The Importance of Motor Skills for Development.

Motor skills are important for development. Everything infants do involves motor skills - postural, locomotor, and manual actions; exploratory actions; social interactions; and actions with artifacts. Put another way, all behavior is motor behavior, and thus motor skill acquisition is synonymous with behavioral development. Age norms for basic motor skills provide useful diagnostics for "typical" development, but cultural differences in child-rearing practices influence skill onset ages. Whenever they emerge, motor skills lay the foundation for development by opening up new opportunities for learning. Postural control brings new parts of the environment into view and into reach; locomotion makes the larger world accessible; manual skills promote new forms of interactions with objects; and motor skills involving every part of the body enhance opportunities for social interaction. Thus, motor skills can instigate a cascade of developments in domains far afield from motor behavior - perception and cognition, language and communication, emotional expression and regulation, physical growth and health, and so on. Finally, motor skill acquisition makes behavior increasingly functional and flexible. Infants learn to tailor behavior to variations in their body and environment and to discover or construct new means to achieve their goals.

The positive influence of manual object exploration on predictive grasping for a moving object in 9-month-old infants.

The present study examined whether infants' manual prediction ability is related to different types of their manual object exploration behavior. Thirty-two 9-month-old infants were tested in a manual prediction task, in which they were encouraged to reach for a temporarily occluded moving object. All infants also participated in a manual exploration task, in which they could freely explore five toy blocks. Infants with a high number of haptic scans in the manual exploration task showed a higher prediction rate in the manual prediction task compared to infants with a low haptic scan score. Reaction times of all infants decreased during the test blocks. However, the reaction time of infants with a high haptic scan score was faster in general. Our findings suggest that object experiences gathered by specific manual exploratory actions, such as haptic scans, are related to infants' predictive abilities when reaching and grasping for a temporarily occluded moving object.

Training of 7-month-old infants’ manual object exploration skills: Effects of active and observational experience

In the present study, a fine motor training was developed and evaluated in which infants were trained to manually explore objects in an advanced manner. Fifty 7-month-old infants were randomly assigned to three different training conditions: (1) to an active manual exploration training, in which they learned to explore objects efficiently, (2) to an observational manual exploration training, in which they observed how an adult performed sophisticated actions on objects, or (3) to a control group receiving no training. The results impressively indicate that infants with a low level of object exploration skill prior to the training showed the most training effects as compared to infants with proficient object exploratory actions. Interestingly, this differential training effect was true for both the active and observational training, highlighting the role of social learning in infancy. Importantly, our study shows for the first time the impact of normal individual variation in infants’ manual object exploration skills on the effects of a fine motor training.

Shake it baby, but only when needed: Preschoolers adapt their exploratory strategies to the information structure of the task

Previous research has suggested that active engagement with the world drives children’s remarkable learning capabilities. We investigated whether preschoolers are “ecological learners,” that is, whether they are able to select those active learning strategies that are most informative in a given task. Children had to choose which of two exploratory actions (open vs. shake) to perform to find an egg shaker hidden in one of four small boxes, contained in two larger boxes. Prior to this game, children either learnt that the egg was equally likely to be found in any of the four small boxes (Uniform condition), or that it was most likely to be found in one particular small box (Skewed condition). Results of Study 1 show that 3- and 4-year-olds successfully tailored their exploratory actions to the different likelihood-distributions: They were more likely to shake first in the Uniform compared to the Skewed condition. Five-year-olds were equally likely to shake first, irrespective of condition, even when incentivized to shake only when needed (Study 2a). However, when the relevance of the frequency training for the hiding game was highlighted (Study 2b and Study 2c), the 5-year-olds showed the same behavioural pattern as the younger preschoolers in Study 1. We suggest that ecological learning may be a key mechanism underlying children’s effectiveness in active learning.

Adaptive Fuzzy Watkins: A New Adaptive Approach for Eligibility Traces in Reinforcement Learning

Reinforcement learning is one of the most reliable methods, which have been used to solve many problems. One of the best reinforcement learning family methods are temporal difference methods. The most important weakness of reinforcement learning methods, such as temporal difference methods, is that these methods have slow convergence rate. Many studies are devoted to solving this problem. One of the proposed solutions to this problem is eligibility traces. Owing to the nature of off-policy methods, combining eligibility traces with off-policy methods requires special attention. In the early learning process for Watkins method (one of the dominant eligibility traces methods), cutting eligibility traces during exploratory actions results in diminishing benefits of eligibility traces method. In this study, we propose a framework to combine eligibility traces with off-policy methods. This research attempts to properly use the information explored during action exploration of the agent; to this end, the decision about applying the eligibility traces during the exploratory actions of the agent is made by means of fuzzy adaptation. We apply this method to find the goal state in the static and dynamic grid world. We compare our approach against the state of the art techniques and show that it outperforms these techniques both in terms of averaged achieved reward and also the convergence time.

Social-cognitive predictors of career exploration and decision-making: Longitudinal test of the career self-management model.

We tested the social-cognitive model of career self-management (Lent & Brown, 2013) using a longitudinal design. Participants were 420 college students who completed measures of career exploration and decision-making self-efficacy, outcome expectations, social support, goals, and actions, along with trait conscientiousness, at 2 time points roughly 4 months apart, near the beginning and middle of an academic year. They also reported their level of career decidedness and decisional anxiety at both of these time points as well as near the end of the academic year (about 3 months after the 2nd assessment). The model provided good overall fit to the data and accounted for substantial portions of the variance in engagement in exploratory actions and in the 2 decisional outcomes (decidedness and decisional anxiety) over time. The findings also supported the hypothesized bidirectional paths of the decisional outcomes to self-efficacy and outcome expectations. We consider implications of the findings for the social-cognitive model as well as for future research and practice. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Feedback and Guidance to Support Children with Intellectual Disabilities in Discovery Learning with a Tangible Interactive Tabletop

By coupling dynamic digital representations to physical objects, tangible environments expand the possibilities for providing scaffolding in discovery learning processes. This is particularly promising in the case of children with intellectual disabilities, for whom the typical lack of structure of discovery-based activities is challenging. In this research, empirical studies were undertaken where children with intellectual disabilities engaged in a process of discovery using a tangible interactive tabletop, which provided a substantial amount of feedback. Two conditions of human mediation (free and guided exploration) were compared, to identify which one enabled more opportunities for discovery, in combination with design aspects of system feedback. Episodes of discovery were found to be much more frequent than episodes of disruption in both conditions, confirming the potential of tangible tabletops, in particular the importance of informational system feedback to learners' exploratory actions. Nevertheless, human mediation is still needed for children with intellectual disabilities despite the feedback capabilities of tangible environments. Although children explored more in free sessions (in terms of amount of exploratory actions), guided exploration produced more episodes of discovery, reinforcing the importance of external guidance for increasing the attention to learning concepts.

Control of a bioreactor using a new partially supervised reinforcement learning algorithm

In recent years, researchers have explored the application of Reinforcement Learning (RL) and Artificial Neural Networks (ANNs) to the control of complex nonlinear and time varying industrial processes. However RL algorithms use exploratory actions to learn an optimal control policy and converge slowly while popular inverse model ANN based control strategies require extensive training data to learn the inverse model of complex nonlinear systems. In this paper a novel approach that avoids the need for extensive training data to construct an exact inverse model in the inverse ANN approach, the need for an exact and stable inverse to exist and the need for exhaustive and costly exploration in pure RL based strategies is proposed. In this approach an initial approximate control policy learnt by an artificial neural network is refined using a reinforcement learning strategy. This Partially Supervised Reinforcement Learning (PSRL) strategy is applied to the economically important problem of control of a semi-continuous batch-fed bioreactor used for yeast fermentation. The bioreactor control problem is formulated as a Markov Decision Process (MDP) and solved using pure RL and PSRL algorithms. Model based and model-free RL control experiments and simulations are used to demonstrate the superior performance of the PSRL strategy compared to the pure RL and inverse model ANN based control strategies on a variety of performance metrics.

Model-Plant Mismatch Compensation Using Reinforcement Learning

Learning-based approaches are suitable for the control of systems with unknown dynamics. However, learning from scratch involves many trials with exploratory actions until a good control policy is discovered. Real robots usually cannot withstand the exploratory actions and suffer damage. This problem can be circumvented by combining learning with a model-based control. In this letter, we employ a nominal model-predictive controller that is impeded by the presence of an unknown model-plant mismatch. To compensate for the mismatch, we propose two approaches of combining reinforcement learning with the nominal controller. The first approach learns a compensatory control action that minimizes the same performance measure as is minimized by the nominal controller. The second approach learns a compensatory signal from a difference of a transition predicted by the internal model and an actual transition. We compare the approaches on a robot attached to the ground and performing a setpoint reaching task in simulations. We implement the better approach on the real robot and demonstrate successful learning results.

Development and Validation of a Sensor-Based Algorithm for Detecting the Visual Exploratory Actions

Wearable sensors are becoming widely used in the sport sciences to assess the performance of athletes. Advances in microelectromechanical systems technology, in particular inertial measurement units (IMUs), provide researchers and practitioners with a portable means of capturing performance in representative task scenarios. Of recent interest to sport scientists in team sports is how athletes perceive their surroundings and how visual (and other) information is used to select appropriate actions during a match. Collectively, the movements athletes make to gather information from their environments is referred to as exploratory action. An important aspect of this behavior is typically measured by notating (counting) the number of head turns from a third-person video perspective. A notational analysis is a labor-intensive task and prone to human error, especially when activity takes place over long durations. The IMUs are well suited to resolve these issues; they are highly accurate, very efficient, and have an adequate output data rate. Currently, no gold standard method exists to automatically detect head turn events from the IMUs. In the current study, a novel algorithm that utilizes data captured from a head-mounted IMU to count the number of head turns performed by an athlete during a controlled experimental task is presented. Results demonstrate that the presented algorithm is an appropriate and efficient method for assessing the number of head turns as a measure of exploratory actions.