Tasks In Dynamic Environment Research Articles

Character navigation in dynamic environments based on optical flow

AbstractSteering and navigation are important components of character animation systems to enable them to autonomously move in their environment. In this work, we propose a synthetic vision model that uses visual features to steer agents through dynamic environments. Our agents perceive optical flow resulting from their relative motion with the objects of the environment. The optical flow is then segmented and processed to extract visual features such as the focus of expansion and time‐to‐collision. Then, we establish the relations between these visual features and the agent motion, and use them to design a set of control functions which allow characters to perform object‐dependent tasks, such as following, avoiding and reaching. Control functions are then combined to let characters perform more complex navigation tasks in dynamic environments, such as reaching a goal while avoiding multiple obstacles. Agent's motion is achieved by local minimization of these functions. We demonstrate the efficiency of our approach through a number of scenarios. Our work sets the basis for building a character animation system which imitates human sensorimotor actions. It opens new perspectives to achieve realistic simulation of human characters taking into account perceptual factors, such as the lighting conditions of the environment.

A high accurate vehicle speed estimation method

In this paper, we present a novel approach for accurate vehicle speed estimation from video sequences. Common methods usually track sets of distinguishing features; however, feature extraction is a difficult task in dynamic environments. Herein, we propose a novel analysis method without feature extraction. Initially, a frame difference method is applied to a region of interest, from which projection histograms are obtained and a group of key bins are selected to represent the vehicle motion. Then, all the possible speeds are tested one by one, and the extreme value of the testing function is selected for the corresponding speed. The proposed system was tested on three data sets containing 2054 vehicles, where the ground truth of speed is obtained by a radar speed detector. The experiment results show that the proposed system has an average error of 0.3 km/h, with 99.4% of the estimation speed within the error of range (− 2 km/h, 2 km/h). The system turns out to be robust, accurate and real time for practical use.

The +1 Method: Model-Free Adaptive Repositioning Policies for Robotic Multi-Agent Systems

Robotic multi-agent systems can efficiently handle spatially distributed tasks in dynamic environments. Problem instances of particular interest, and generality are the dynamic traveling repairman problem, and the dynamic vehicle routing problem. Operational policies for robotic fleets solving these two problems take decisions in an online setting with continuously arriving demands to optimize service level, and efficiency, and can be classified along several lines. First, some require a model of the demand, e.g., based on historical information, while others work model-free. Second, they are designed for different operating conditions from light to heavy system load. Third, they work in a time-invariant or time-varying setting. We present a novel class of model-free operational policies for time-varying demands, which have performance independent of the load factor, a combination of properties not achieved by other operational policies in the literature. The underlying principle of the introduced policies is to send available robots to recent service request locations. In simple terms, they rely on sending more than one robot for every service request arriving to the system. This leads to an advantage in scenarios where demand is non-uniformly distributed, and correlated in space, and time. We provide performance guarantees for both the time-invariant, and the time-varying cases as well as for correlated demand. We verify our theoretical results numerically. Finally, we apply our operational policy to the problem of mobility-on-demand fleet operation, and demonstrate that it outperforms model-based, and complex algorithms across all load ranges, despite its simplicity.

Shared right-hemispheric representations of sensorimotor goals in dynamic task environments.

Functional behaviour affords that we form goals to integrate sensory information about the world around us with suitable motor actions, such as when we plan to grab an object with a hand. However, much research has tested grasping in static scenarios where goals are pursued with repetitive movements, whereas dynamic contexts require goals to be pursued even when changes in the environment require a change in the actions to attain them. To study grasp goals in dynamic environments here, we employed a task where the goal remained the same but the execution of the movement changed; we primed participants to grasp objects either with their right or left hand, and occasionally they had to switch to grasping with both. Switch costs should be minimal if grasp goal representations were used continuously, for example, within the left dominant hemisphere. But remapped or re-computed goal representations should delay movements. We found that switching from right-hand grasping to bimanual grasping delayed reaction times but switching from left-hand grasping to bimanual grasping did not. Further, control experiments showed that the lateralized switch costs were not caused by asymmetric inhibition between hemispheres or switches between usual and unusual tasks. Our results show that the left hemisphere does not serve a general role of sensorimotor grasp goal representation. Instead, sensorimotor grasp goals appear to be represented at intermediate levels of abstraction, downstream from cognitive task representations, yet upstream from the control of the grasping effectors.

Meta-Seg: A Generalized Meta-Learning Framework for Multi-Class Few-Shot Semantic Segmentation

Semantic segmentation performs pixel-wise classification for given images, which can be widely used in autonomous driving, robotics, medical diagnostics and etc. The recent advanced approaches have witnessed rapid progress in semantic segmentation. However, these supervised learning based methods rely heavily on large-scale datasets to acquire strong generalizing ability, such that they are coupled with some constraints. Firstly, human annotation of pixel-level segmentation masks is laborious and time-consuming, which causes relatively expensive training data and make it hard to deal with urgent tasks in dynamic environment. Secondly, the outstanding performance of the above data-hungry methods will decrease with few available training examples. In order to overcome the limitations of the supervised learning semantic segmentation methods, this paper proposes a generalized meta-learning framework, named Meta-Seg. It consists of a meta-learner and a base-learner. Specifically, the meta-learner learns a good initialization and a parameter update strategy from a distribution of few-shot semantic segmentation tasks. The base-learner can be any semantic segmentation models theoretically and can implement fast adaptation (that is updating parameters with few iterations) under the guidance of the meta-learner. In this work, the successful semantic segmentation model FCN8s is integrated into Meta-Seg. Experiments on the famous few-shot semantic segmentation dataset PASCAL $5^{i}$ prove Meta-Seg is a promising framework for few-shot semantic segmentation. Besides, this method can provide with reference for the relevant researches of meta-learning semantic segmentation.

Measuring Workload in a Multitasking Environment Using Fractal Dimension of Pupil Dilation

ABSTRACTThe purpose of this study is to investigate the usefulness of the fractal dimension of pupil dilation as a measure of workload in a multitasking environment. Researchers have found that pupil dilation can reveal the underlying mechanism of the cognitive workload in a dynamic task environment. However, the workload metrics by using pupillary responses in a multitasking environment are not well studied in the literature. In this study, the participants’ pupil dilation was collected and analyzed by using a fractal analysis technique to assess the participants’ workload during the experiment. During the experiment, each participant performed a process monitoring task and Multi-Attribute Task Battery (MATB) task together. The process monitoring task simulated the responsibilities of process operators in the control room of oil and gas refinery plants. The MATB task consisted of system monitoring, target tracking, and dynamic resource management. To validate the outcome of this new metric, NASA-TLX questionnaires were used to measure the subjective workload, and the result of NASA-TLX was compared to the fractal dimension result. The findings from this study showed that the fractal dimension of pupil dilation could be used as a new physiological index to measure the workload in a multitasking environment.

Dyadic Team Interaction and Shared Cognition to Inform Human-Robot Teaming

Project overview. The current research aims to understand how human operators effectively team with urban search robot teammates in a dynamic and complex task environment. With that in mind, we examined how shared cognition and restricted language capabilities impact performance of human dyadic teams using a simulated Minecraft task environment. In this human dyadic team, an internal teammate (comparable to robot) identifies the location of victims while navigating inside a game environment that reflects a collapsed building; and an external teammate (comparable to operator) sees their teammate's actions from a different screen and guides them through the environment, tracking the location of victims on a map as they go. In order to examine the effects of language and shared cognition, a two by two design was chosen: (1) in the communication manipulation, participants are either able to communicate using natural language or the internal participant’s communication is limited to three-word utterances; and (2) in the shared cognition manipulation, either the internal participant is made fully aware of the external participant’s restricted representation of the environment and inaccurate map or the internal is unaware of these challenges. Method. This study used a simulated search task, in the Minecraft gaming environment, where two humans acted as a human-robot search team (Bartlett & Cooke, 2015). After signing consent forms, two randomly selected participants completed a half-hour training session for their individual role. Both participants sat in the same room but had a divider between them, and then, interacted to complete a 15-minute simulated search task. The Minecraft environment resembled an office structure with interspersed green, blue, and pink blocks that represented potential targets. Blue and green blocks were meant to represent survivors, whereas pink blocks represented hazards. Pressing a button on green blocks counted positively towards the team’s performance, whereas pressing a button on pink blocks counted negatively towards the team’s performance. Blue blocks were time-sensitive, such that pressing a button on them before eight minutes into the scenario counted positively toward performance, but pressing the button after that time counted against performance. Pressing the button on any block more than once counted negatively towards the team’s performance. A map of this environment was also made available. Inconsistencies were intentionally introduced, such as missing walls, additional walls, and misplaced doorways to simulate a damaged building, none of which were depicted on the map. Due to the dynamic nature of the task, effective communication and coordination between the dyads is required for effective performance. Several measures were obtained in this research: team performance, situation awareness, NASA TLX workload, team verbal behaviors, team communication flow, and demographics. In the interest of space, we only present team performance, a determinism measure (served as an index of flexible behavior and was estimated from team communication flow, using Recurrence Quantification Analysis (Marwan, Carmen Romano, Thiel, & Kurths, 2007), and NASA TLX workload. Results and conclusion. The primary findings from this study are that: 1) teams in the natural language and shared model conditions, performed better than teams with the limited language and restricted model, respectively; 2) when the internal participant is unaware of the challenges of the external, the external perceives higher workload than when there is a shared cognition; 3) teams with natural language and shared model demonstrated more predictable behavior than the other teams; 4) some amount of systems predictability is good but too much predictability is not good in the system – this also confirms another study: Demir, Likens, Cooke, Amazeen, & McNeese, InReview. Overall these results indicate that effective team interaction and shared cognition play an important role in human-robot teaming performance. Acknowledgements. Human-Robot Dyad research was partially supported by ONR Grant N0014-13-1-0519 to PI: Subbarao Kambhampati (Program Managers: Marc Steinberg). We also acknowledge assistance of data collection Aaron Bradbury, Emily Gran, Jocelyn Martinez, and Madeline Niichel.

Causal Evidence for Learning-Dependent Frontal Lobe Contributions to Cognitive Control.

The lateral prefrontal cortex (LPFC) plays a central role in the prioritization of sensory input based on task relevance. Such top-down control of perception is of fundamental importance in goal-directed behavior, but can also be costly when deployed excessively, necessitating a mechanism that regulates control engagement to align it with changing environmental demands. We have recently introduced the "flexible control model" (FCM), which explains this regulation as resulting from a self-adjusting reinforcement-learning mechanism that infers latent statistical structure in dynamic task environments to predict forthcoming states. From this perspective, LPFC-based control is engaged as a function of anticipated cognitive demand, a notion for which we previously obtained correlative neuroimaging evidence. Here, we put this hypothesis to a rigorous, causal test by combining the FCM with a transcranial magnetic stimulation (TMS) intervention that transiently perturbed the LPFC. Human participants (male and female) completed a nonstationary version of the Stroop task with dynamically changing probabilities of conflict between task-relevant and task-irrelevant stimulus features. TMS was given on each trial before stimulus onset either over the LPFC or over a control site. In the control condition, we observed adaptive performance fluctuations consistent with demand predictions that were inferred from recent and remote trial history and effectively captured by our model. Critically, TMS over the LPFC eliminated these fluctuations while leaving basic cognitive and motor functions intact. These results provide causal evidence for a learning-based account of cognitive control and delineate the nature of the signals that regulate top-down biases over stimulus processing.SIGNIFICANCE STATEMENT A core function of the human prefrontal cortex is to control the signal flow in sensory brain regions to prioritize processing of task-relevant information. Abundant work suggests that such control is flexibly recruited to accommodate dynamically changing environmental demands, yet the nature of the signals that serve to engage control remains unknown. Here, we combined computational modeling with noninvasive brain stimulation to show that changes in control engagement are captured by a self-adjusting reinforcement-learning mechanism that tracks changing environmental statistics to predict forthcoming processing demands and that transient perturbation of the prefrontal cortex abolishes these adjustments. These findings delineate the learning signals that underpin adaptive engagement of prefrontal control functions and provide causal evidence for their relevance in behavioral control.

Towards a common implementation of reinforcement learning for multiple robotic tasks

Mobile robots are increasingly being employed for performing complex tasks in dynamic environments. Those tasks can be either explicitly programmed by an engineer or learned by means of some automatic learning method, which improves the adaptability of the robot and reduces the effort of setting it up. In this sense, reinforcement learning (RL) methods are recognized as a promising tool for a machine to learn autonomously how to do tasks that are specified in a relatively simple manner. However, the dependency between these methods and the particular task to learn is a well-known problem that has strongly restricted practical implementations in robotics so far. Breaking this barrier would have a significant impact on these and other intelligent systems; in particular, having a core method that requires little tuning effort for being applicable to diverse tasks would boost their autonomy in learning and self-adaptation capabilities. In this paper we present such a practical core implementation of RL, which enables the learning process for multiple robotic tasks with minimal per-task tuning or none. Based on value iteration methods, we introduce a novel approach for action selection, called Q-biased softmax regression (QBIASSR), that takes advantage of the structure of the state space by attending the physical variables involved (e.g., distances to obstacles, robot pose, etc.), thus experienced sets of states accelerate the decision-making process of unexplored or rarely-explored states. Intensive experiments with both real and simulated robots, carried out with the software framework also introduced here, show that our implementation is able to learn different robotic tasks without tuning the learning method. They also suggest that the combination of true online SARSA(λ) (TOSL) with QBIASSR can outperform the existing RL core algorithms in low-dimensional robotic tasks. All of these are promising results towards the possibility of learning much more complex tasks autonomously by a robotic agent.

Manipulation of Biomolecule‐Modified Liquid‐Metal Blobs

AbstractSoft and deformable liquid metals (LMs) are building components in various systems related to uncertain and dynamic task environments. Herein we describe the development of a biomolecule‐triggered external‐manipulation method involving LM conjugates for the construction of future innovative soft robotics operating in physiological environments. Functional soft hybrids composed of a liquid‐metal droplet, a thiolated ligand, and proteins were synthesized for the expression of diverse macroscopic commands, such as attachment to cells, binary fusion, and self‐propelled movement through molecular recognition and enzymatic reactions. Our technology could be used to create new state‐of‐the‐art soft robots for chemical and biomedical engineering applications.

Manipulation of Biomolecule-Modified Liquid-Metal Blobs.

Soft and deformable liquid metals (LMs) are building components in various systems related to uncertain and dynamic task environments. Herein we describe the development of a biomolecule-triggered external-manipulation method involving LM conjugates for the construction of future innovative soft robotics operating in physiological environments. Functional soft hybrids composed of a liquid-metal droplet, a thiolated ligand, and proteins were synthesized for the expression of diverse macroscopic commands, such as attachment to cells, binary fusion, and self-propelled movement through molecular recognition and enzymatic reactions. Our technology could be used to create new state-of-the-art soft robots for chemical and biomedical engineering applications.

Response-cue interval effects in extended-runs task switching: memory, or monitoring?

This study investigated effects of manipulating the response-cue interval (RCI) in the extended-runs task-switching procedure. In this procedure, a task cue is presented at the start of a run of trials and then withdrawn, such that the task has to be stored in memory to guide performance until the next task cue is presented. The effects of the RCI manipulation were not as predicted by an existing model of memory processes in task switching (Altmann and Gray, Psychol Rev 115:602-639, 2008), suggesting that either the model is incorrect or the RCI manipulation did not have the intended effect. The manipulation did produce a theoretically meaningful pattern, in the form of a main effect on response time that was not accompanied by a similar effect on the error rate. This pattern, which replicated across two experiments, is interpreted here in terms of a process that monitors for the next task cue, with a longer RCI acting as a stronger signal that a cue is about to appear. The results have implications for the human factors of dynamic task environments in which critical events occur unpredictably.

Effects of agent timing on the human-agent team

As technology becomes more sophisticated, autonomous agents are applied more frequently to improve system performance. The current research employed a five step method, including modeling, simulation, and human experimentation to explore the effect of an artificial agent’s timing on the performance of a human-agent team within a highly dynamic task environment. Agent timing significantly influenced the role assumed by the human within the team. Further, agent timing changed system performance by approximately 40% within the experimental conditions. Results indicate that an artificial agent’s timing can be varied as a function of the task demands placed upon the human-agent team to maintain an appropriate level of human activity and engagement. Therefore, agent timing may be controlled to adapt autonomy to provide an apparent continuum along which to control human engagement in systems employing human-agent teaming within dynamic environments.

Laser-Based Bidirectional Pedestrian Counting via Height Map Guided Regression and Voting

Pedestrian counting is valuable for business decisions and public safety, but it remains a challenging task in dynamic environments with illumination changes, shadows, background clutter, etc. In this paper, we investigate a novel laser-based bidirectional pedestrian counting approach that addresses these problems effectively. In the proposed method, a laser range scanner is employed in a bird’s eye view to measure the distances between the laser range scanner and obstacles below. Height data of passing pedestrians are accumulated from each scan to generate a height map with noise eliminated. In order to count the number of pedestrians walking in close distances, we apply linear regression to estimate the number of pedestrians in a height cluster using several geometrical features. A variable-sized slide window is then utilized to detect heads in height data. Furthermore, the direction of each pedestrian is determined by voting results of scanning points in the detected head. Experimental results in different environments demonstrate the accuracy and efficiency of our method.

Improving Physical Task Performance with Counterfactual and Prefactual Thinking.

Counterfactual thinking (reflecting on “what might have been”) has been shown to enhance future performance by translating information about past mistakes into plans for future action. Prefactual thinking (imagining “what might be if…”) may serve a greater preparative function than counterfactual thinking as it is future-orientated and focuses on more controllable features, thus providing a practical script to prime future behaviour. However, whether or not this difference in hypothetical thought content may translate into a difference in actual task performance has been largely unexamined. In Experiment 1 (n = 42), participants performed trials of a computer-simulated physical task, in between which they engaged in either task-related hypothetical thinking (counterfactual or prefactual) or an unrelated filler task (control). As hypothesised, prefactuals contained more controllable features than counterfactuals. Moreover, participants who engaged in either form of hypothetical thinking improved significantly in task performance over trials compared to participants in the control group. The difference in thought content between counterfactuals and prefactuals, however, did not yield a significant difference in performance improvement. Experiment 2 (n = 42) replicated these findings in a dynamic balance task environment. Together, these findings provide further evidence for the preparatory function of counterfactuals, and demonstrate that prefactuals share this same functional characteristic.

Interrogating Feature Learning Models to Discover Insights Into the Development of Human Expertise in a Real-Time, Dynamic Decision-Making Task.

Tetris provides a difficult, dynamic task environment within which some people are novices and others, after years of work and practice, become extreme experts. Here we study two core skills; namely, (a) choosing the goal or objective function that will maximize performance and (b)a feature-based analysis of the current game board to determine where to place the currently falling zoid (i.e., Tetris piece) so as to maximize the goal. In Study 1, we build cross-entropy reinforcement learning (CERL) models (Szita & Lorincz, 2006) to determine whether different goals result in different feature weights. Two of these optimization strategies quickly rise to performance plateaus, whereas two others continue toward higher but more jagged (i.e., variable) heights. In Study 2, we compare the zoid placement decisions made by our best CERL models with those made by 67 human players. Across 370,131 human game episodes, two CERL models picked the same zoid placements as our lowest scoring human for 43% of the placements and as our three best scoring experts for 65% of the placements. Our findings suggest that people focus on maximizing points, not number of lines cleared or number of levels reached. They also show that goal choice influences the choice of zoid placements for CERLs and suggest that the same is true of humans. Tetris has a repetitive task structure that makes Tetris more tractable and more like a traditional experimental psychology paradigm than many more complex games or tasks. Hence, although complex, Tetris is not overwhelmingly complex and presents a right-sized challenge to cognitive theories, especially those of integrated cognitive systems.

Design and application of a CA-BDI model to determine farmers' land-use behavior.

The belief-desire-intention (BDI) model has been widely used to construct reasoning systems for complex tasks in dynamic environments. We have designed a capabilities and abilities (CA)-BDI farmer decision-making model, which is an extension of the BDI architecture and includes internal representations for farmer household Capabilities and Abilities. This model is used to explore farmer learning mechanisms and to simulate the bounded rational decisions made by farmer households. Our case study focuses on the Gaoqu Commune of Mizhi County, Shaanxi Province, China, where scallion is one of the main cash crops. After comparing the differences between actual land-use changes from 2007 to 2009 and the simulation results, we analyze the validity of the model and discuss the potential and limitations of the farmer land-use decision-making model under three scenarios. Based on the design and implementation of the model, the following conclusions can be drawn: (1) the CA-BDI framework is an appropriate model for exploring learning mechanisms and simulating bounded rational decisions; and (2) local governments should encourage scallion planting by assisting scallion farmer cooperatives and farmers to understand the market risk, standardize the rules of their cooperation, and supervise the contracts made between scallion cooperatives and farmers.

Classification systems in dynamic environments: an overview

Data mining and machine learning algorithms can be employed to perform a variety of tasks. However, since most of these problems may depend on environments that change over time, performing classification tasks in dynamic environments has been a challenge in data mining research domain in the last decades. Currently, in the literature, the most common strategies used to detect changes are based on accuracy monitoring, which relies on previous knowledge of the data in order to identify whether or not correct classifications are provided. However, such a feedback can be infeasible in practical problems. In this work, we present a comprehensive overview of current machine learning/data mining approaches proposed to deal with dynamic environments problems. The objective is to highlight the main drawbacks and open issues, as well as future directions and problems worthy of investigation. In addition, we provide the definitions of the main terms used to represent this problem in the literature, such as concept drift and novelty detection. WIREs Data Mining Knowl Discov 2016, 6:156–166. doi: 10.1002/widm.1184This article is categorized under: Technologies > Classification Technologies > Machine Learning

Cooperation in Human-Agent Systems to Support Resilience: A Microworld Experiment.

This study uses a dyadic approach to understand human-agent cooperation and system resilience. Increasingly capable technology fundamentally changes human-machine relationships. Rather than reliance on or compliance with more or less reliable automation, we investigate interaction strategies with more or less cooperative agents. A joint-task microworld scenario was developed to explore the effects of agent cooperation on participant cooperation and system resilience. To assess the effects of agent cooperation on participant cooperation, 36 people coordinated with a more or less cooperative agent by requesting resources and responding to requests for resources in a dynamic task environment. Another 36 people were recruited to assess effects following a perturbation in their own hospital. Experiment 1 shows people reciprocated the cooperative behaviors of the agents; a low-cooperation agent led to less effective interactions and less resource sharing, whereas a high-cooperation agent led to more effective interactions and greater resource sharing. Experiment 2 shows that an initial fast-tempo perturbation undermined proactive cooperation-people tended to not request resources. However, the initial fast tempo had little effect on reactive cooperation-people tended to accept resource requests according to cooperation level. This study complements the supervisory control perspective of human-automation interaction by considering interdependence and cooperation rather than the more common focus on reliability and reliance. The cooperativeness of automated agents can influence the cooperativeness of human agents. Design and evaluation for resilience in teams involving increasingly autonomous agents should consider the cooperative behaviors of these agents.

A guideline for integrating dynamic areas of interests in existing set-up for capturing eye movement: Looking at moving aircraft.

Today, capturing the behavior of a human eye is considered a standard method for measuring the information-gathering process and thereby gaining insights into cognitive processes. Due to the dynamic character of most task environments there is still a lack of a structured and automated approach for analyzing eye movement in combination with moving objects. In this article, we present a guideline for advanced gaze analysis, called IGDAI (Integration Guideline for Dynamic Areas of Interest). The application of IGDAI allows gathering dynamic areas of interest and simplifies its combination with eye movement. The first step of IGDAI defines the basic requirements for the experimental setup including the embedding of an eye tracker. The second step covers the issue of storing the information of task environments for the dynamic AOI analysis. Implementation examples in XML are presented fulfilling the requirements for most dynamic task environments. The last step includes algorithms to combine the captured eye movement and the dynamic areas of interest. A verification study was conducted, presenting an air traffic controller environment to participants. The participants had to distinguish between different types of dynamic objects. The results show that in comparison to static areas of interest, IGDAI allows a faster and more detailed view on the distribution of eye movement.