Goal Location Research Articles

A Hippocampal-parietal Network for Reference Frame Coordination.

Navigating space and forming memories based on spatial experience are crucial for survival, including storing memories in an allocentric (map-like) framework and conversion into egocentric (body-centered) action. The hippocampus and parietal cortex (PC) comprise a network for coordinating these reference frames, though the mechanism remains unclear. We used a task requiring remembering previous spatial locations to make correct future action and observed that hippocampus can encode the allocentric place, while PC encodes upcoming actions and relays this to hippocampus. Transformation from location to action unfolds gradually, with 'Came From' signals diminishing and future action representations strengthening. PC sometimes encodes previous spatial locations in a route-based reference frame and conveys this to hippocampus. The signal for the future location appears first in PC, and then in hippocampus, in the form of an egocentric direction of future goal locations, suggesting egocentric encoding recently observed in hippocampus may originate in PC (or another "upstream" structure). Bidirectional signaling suggests a coordinated mechanism for integrating allocentric, route-centered, and egocentric spatial reference frames at the network level during navigation.Significance Statement Our study has broad implications for understanding how the brain coordinates and integrates different spatial reference frames. Our data suggests rats can alternate between multiple neural strategies within the same task. In addition, we find out that similar signals are present in both the hippocampus and the PC but at different times, providing novel insights into the mechanisms underlying spatial navigation and memory. It reveals an intricate system involving an extended brain network that includes the hippocampus, PC, and structures anatomically 'in-between'. The bidirectional signaling suggests this brain network truly operates as a network and not a unidirectional circuit. These findings suggest a focus on brain networks (and not just single regions) is critical for understanding transformations. These processes are fundamental for spatial cognition and related disorders and even for solving problems that may use similar neural machinery, such as building abstract representations from egocentric views, as occurs with object invariance.

Mobile Robots for Environment-Aware Navigation: A Code-Free Approach with Topometric Maps for Non-Expert Users

The growing use of mobile robots in unconventional environments demands new programming approaches to make them accessible to non-expert users. Indeed, traditional programming methods require specialized expertise in robotics and programming, limiting robots’ accessibility to a broader audience. End-user robot programming has emerged to overcome these limitations, aiming to simplify robot programming through intuitive methods. In this work, we propose a code-free approach for programming mobile robots to autonomously execute navigation tasks, i.e., reach a desired goal location from an arbitrary initial position. Our method relies on instructing the robot on new paths through demonstrations while creating and continuously updating a topometric map of the environment. Moreover, by leveraging the information gathered during the instruction phase, the robot can perceive slight environmental changes and autonomously make the best decision in response to unexpected situations (e.g., adjusting its path, stopping, or requesting user intervention). Experiments conducted in both simulated and real-world environments support the validity of our approach, as they show that the robot can successfully reach its assigned goal location in the vast majority of cases.

Wild Tibetan Macaques Use a Route-Based Mental Map to Navigate in Large-Scale Space.

Many animals face significant challenges in locating and acquiring resources that are unevenly distributed in space and time. In the case of nonhuman primates, it remains unclear how individuals remember goal locations and whether they navigate using a route-based or a coordinate-based mental representation when moving between out-of-sight feeding and resting sites (i.e., large-scale space). Here, we examine spatial memory and mental map formation in wild Tibetan macaques (Macaca thibetana) inhabiting a mountainous, forested ecosystem characterized by steep terrain that limits direct vision to 25 meters. We used an instantaneous scan sampling technique at 10-min intervals to record the behavior and location of macaques on Mt. Huangshan, Anhui Province, China, from September 2020 to August 2023. Over 214 days, we obtained 7180 GPS points of the macaques' locations. Our study revealed that the macaques reused 1264 route segments (average length 204.26 m) at least four times each. The number of feeding and resting sites around the habitual route segment, terrain roughness, and dense vegetation areas significantly influenced the use of route segments by our study group. In addition, we found evidence that the monkeys reused 48 nodes to reorient their travel path. We found that monkeys approached a revisited foraging or resting site from the same limited set of directions, which is inconsistent with a coordinate-based spatial representation. In addition, the direction in which the macaques left a feeding or resting site was significantly different from the straight-line direction required to reach their next feeding or resting site, suggesting that the macaques frequently reoriented their direction of travel to reach their goal. Finally, on average, macaques traveled 24% (CI = 1.24) farther than the straight-line distance to reach revisited feeding and resting sites. From our robust data set, we conclude that Tibetan macaques navigate large spaces using a route-based mental representation that appears to help them locate food resources in dense, rugged montane forests and heterogeneous habitats.

A cellular basis for mapping behavioural structure

To flexibly adapt to new situations, our brains must understand the regularities in the world, as well as those in our own patterns of behaviour. A wealth of findings is beginning to reveal the algorithms that we use to map the outside world1, 2, 3, 4, 5–6. However, the biological algorithms that map the complex structured behaviours that we compose to reach our goals remain unknown. Here we reveal a neuronal implementation of an algorithm for mapping abstract behavioural structure and transferring it to new scenarios. We trained mice on many tasks that shared a common structure (organizing a sequence of goals) but differed in the specific goal locations. The mice discovered the underlying task structure, enabling zero-shot inferences on the first trial of new tasks. The activity of most neurons in the medial frontal cortex tiled progress to goal, akin to how place cells map physical space. These ‘goal-progress cells’ generalized, stretching and compressing their tiling to accommodate different goal distances. By contrast, progress along the overall sequence of goals was not encoded explicitly. Instead, a subset of goal-progress cells was further tuned such that individual neurons fired with a fixed task lag from a particular behavioural step. Together, these cells acted as task-structured memory buffers, implementing an algorithm that instantaneously encoded the entire sequence of future behavioural steps, and whose dynamics automatically computed the appropriate action at each step. These dynamics mirrored the abstract task structure both on-task and during offline sleep. Our findings suggest that schemata of complex behavioural structures can be generated by sculpting progress-to-goal tuning into task-structured buffers of individual behavioural steps.

Intelligent mobile robot navigation in unknown and complex environment using reinforcement learning technique

The usage of mobile robots (MRs) has expanded dramatically in the last several years across a wide range of industries, including manufacturing, surveillance, healthcare, and warehouse automation. To ensure the efficient and safe operation of these MRs, it is crucial to design effective control strategies that can adapt to changing environments. In this paper, we propose a new technique for controlling MRs using reinforcement learning (RL). Our approach involves mathematical model generation and later training a neural network (NN) to learn a policy for robot control using RL. The policy is learned through trial and error, where MR explores the environment and receives rewards based on its actions. The rewards are designed to encourage the robot to move towards its goal while avoiding obstacles. In this work, a deep Q-learning (QL) agent is used to enable robots to autonomously learn to avoid collisions with obstacles and enhance navigation abilities in an unknown environment. When operating MR independently within an unfamiliar area, a RL model is used to identify the targeted location, and the Deep Q-Network (DQN) is used to navigate to the goal location. We evaluate our approach using a simulation using the Epsilon-Greedy algorithm. The results show that our approach outperforms traditional MR control strategies in terms of both efficiency and safety.

Dopamine-mediated formation of a memory module in the nucleus accumbens for goal-directed navigation.

Spatial memories guide navigation efficiently toward desired destinations. However, the neuronal and circuit mechanisms underlying the encoding of goal locations and its translation into goal-directed navigation remain unclear. Here we demonstrate that mice rapidly form a spatial memory of a shelter during shelter experiences, guiding escape behavior toward the goal location-a shelter-when under threat. Dopaminergic neurons in the ventral tegmental area and their projection to the nucleus accumbens (NAc) encode safety signals associated with the shelter. Optogenetically induced phasic dopamine signals are sufficient to create a place memory that directs escape navigation. Converging dopaminergic and hippocampal glutamatergic inputs to the NAc mediate the formation of a goal-related memory within a subpopulation of NAc neurons during shelter experiences. Artificial co-activation of this goal-related NAc ensemble with neurons in the dorsal periaqueductal gray was sufficient to trigger memory-guided, rather than random, escape behavior. These findings provide causal evidence of cognitive circuit modules linking memory with goal-directed action.

FOS mapping reveals two complementary circuits for spatial navigation in mouse

Here, we show that during continuous navigation in a dynamic external environment, mice are capable of developing a foraging strategy based exclusively on changing distal (allothetic) information and that this process may involve two alternative components of the spatial memory circuit: the hippocampus and retrosplenial cortex. To this end, we designed a novel custom apparatus and implemented a behavioral protocol based on the figure-8-maze paradigm with two goal locations associated with distinct contexts. We assessed whether mice are able to learn to retrieve a sequence of rewards guided exclusively by the changing context. We found out that training mice in the apparatus leads to change in strategy from the internal tendency to alternate into navigation based exclusively on visual information. This effect could be achieved using two different training protocols: prolonged alternation training, or a flexible protocol with unpredictable turn succession. Based on the c-FOS mapping we also provide evidence of opposing levels of engagement of hippocampus and retrosplenial cortex after training of mice in these two different regimens. This supports the hypothesis of the existence of parallel circuits guiding spatial navigation, one based on the well-described hippocampal representation, and another, RSC-dependent.

Time cells in the retrosplenial cortex.

The retrosplenial cortex (RSC) is a key component of the brain's memory systems, with anatomical connections to the hippocampus, anterior thalamus, and entorhinal cortex. This circuit has been implicated in episodic memory and many of these structures have been shown to encode temporal information, which is critical for episodic memory. For example, hippocampal time cells reliably fire during specific segments of time during a delay period. Although RSC lesions are known to disrupt temporal memory, time cells have not been observed there. In this study, we reanalyzed archival RSC neuronal firing data during the intertrial delay period from two previous experiments involving different behavioral tasks, a blocked alternation task and a cued T-maze task. For the blocked alternation task, rats were required to approach the east or west arm of a plus maze for reward during different blocks of trials. Because the reward locations were not cued, the rat had to remember the goal location for each trial. In the cued T-maze task, the reward location was explicitly cued with a light and the rats simply had to approach the light for reward, so there was no requirement to hold a memory during the intertrial delay. Time cells were prevalent in the blocked alternation task, and most time cells clearly differentiated the east and west trials. We also found that RSC neurons could exhibit off-response time fields, periods of reliably inhibited firing. Time cells were also observed in the cued T-maze, but they were less prevalent and they did not differentiate left and right trials as well as in the blocked alternation task, suggesting that RSC time cells are sensitive to the memory demands of the task. These results suggest that temporal coding is a prominent feature of RSC firing patterns, consistent with an RSC role in episodic memory.

A Hippocampal-parietal Network for Reference Frame Coordination.

Navigating space and forming memories based on spatial experience are crucial for survival, including storing memories in an allocentric (map-like) framework and conversion into body-centered action. The hippocampus and parietal cortex (PC) comprise a network for coordinating these reference frames, though the mechanism remains unclear. We used a task requiring remembering previous spatial locations to make correct future action and observed that hippocampus can encode the allocentric place, while PC encodes upcoming actions and relays this to hippocampus. Transformation from location to action unfolds gradually, with 'Came From' signals diminishing and future action representations strengthening. PC sometimes encodes previous spatial locations in a route-based reference frame and conveys this to hippocampus. The signal for the future location appears first in PC, and then in hippocampus, in the form of an egocentric direction of future goal locations, suggesting egocentric encoding recently observed in hippocampus may originate in PC (or another "upstream" structure). Bidirectional signaling suggests a coordinated mechanism for integrating map-like, route-centered, and person-centered spatial reference frames at the network level during navigation.

Recording Neural Reward Signals in a Naturalistic Operant Task Using Mobile-EEG and Augmented Reality.

The electrophysiological response to rewards recorded during laboratory tasks has been well documented, yet little is known about the neural response patterns in a more naturalistic setting. Here, we combined a mobile-EEG system with an augmented reality headset to record event-related brain potentials (ERPs) while participants engaged in a naturalistic operant task to find rewards. Twenty-five participants were asked to navigate toward a west or east goal location marked by floating orbs, and once participants reached the goal location, the orb would then signify a reward (5 cents) or no-reward (0 cents) outcome. Following the outcome, participants returned to a start location marked by floating purple rings, and once standing in the middle, a 3 s counter signaled the next trial, for a total of 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked ∼230 ms with a maximal at channel FCz (M = -0.695 μV, ±0.23) and was significantly different than zero (p < 0.01). Participants took ∼3.38 s to reach the goal location and exhibited a general lose-shift (68.3% ±3.5) response strategy and posterror slowing. Overall, these novel findings provide support for the idea that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step toward a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.

Differential functions of the dorsal and intermediate regions of the hippocampus for optimal goal-directed navigation in VR space

Goal-directed navigation requires the hippocampus to process spatial information in a value-dependent manner, but its underlying mechanism needs to be better understood. Here, we investigated whether the dorsal (dHP) and intermediate (iHP) regions of the hippocampus differentially function in processing place and its associated value information. Rats were trained in a place-preference task involving reward zones with different values in a visually rich virtual reality environment where two-dimensional navigation was possible. Rats learned to use distal visual scenes effectively to navigate to the reward zone associated with a higher reward. Inactivation of both dHP and iHP with muscimol altered the efficiency and precision of wayfinding behavior, but iHP inactivation induced more severe damage, including impaired place preference. Our findings suggest that the iHP is more critical for value-dependent navigation toward higher-value goal locations.

Differential functions of the dorsal and intermediate regions of the hippocampus for optimal goal-directed navigation in VR space.

Goal-directed navigation requires the hippocampus to process spatial information in a value-dependent manner, but its underlying mechanism needs to be better understood. Here, we investigated whether the dorsal (dHP) and intermediate (iHP) regions of the hippocampus differentially function in processing place and its associated value information. Rats were trained in a place-preference task involving reward zones with different values in a visually rich virtual reality environment where two-dimensional navigation was possible. Rats learned to use distal visual scenes effectively to navigate to the reward zone associated with a higher reward. Inactivation of both dHP and iHP with muscimol altered the efficiency and precision of wayfinding behavior, but iHP inactivation induced more severe damage, including impaired place preference. Our findings suggest that the iHP is more critical for value-dependent navigation toward higher-value goal locations.

Optimization of 3D Printing While Traveling En Route to Extend Range of Unmanned Aircraft Systems for Multilocation Mission Scenarios

Abstract The nexus of two relatively recent technologies, additive manufacturing and unmanned aircraft systems (UAS), has enabled new and unique capabilities that have only started to be realized in integrated systems. This article explores and quantifies the impact of 3D printing parts for UAS, or entire UAS systems, on an agent platform, while this agent travels to multiple locations as part of a mission objective. The fully printed or enhanced UAS can then be released at launch points farther away from the goal locations. This, in turn, can accelerate mission completion times and reduce travel costs depending upon the ratio between vehicle speed and 3D printing rate. Thousands of scenarios are optimized across the design space to minimize the travel path length for the agent platform as a result of 3D printing en route to the locations of interest. Results indicate that based on the print capability and agent travel speed, an exponential decay in the amount of travel distance of the agent platform occurs. For unity ratios of print speed and agent speed in the considered design space, a decrease of 55% in the total required distance of our agent is observed. This reduction in total travel distance can reduce time, fuel, cost, and other aspects including other environmental and social impacts. A generalized optimization formulation is also presented at the end to enable similar analyses with other en route range-extending technology such as battery charging.

Reactive shepherding along a dynamic path

Shepherding, the task of guiding a herd of autonomous individuals in a desired direction, is an essential skill employed in the herding of animals, crowd control, and evacuation operations. Integrating shepherding capabilities into robots holds promise to perform such tasks with increased efficiency and reduced labor costs. To date, robotic shepherds have only been designed to steer a herd towards a predetermined goal location without constraints on the trajectory. However, the tasks of a sheepdog encompass not only steering the herd but also (i) maintaining the herd within a designated area and (ii) averting dangers, obstacles, or undesirable terrain such as newly sown land. We present a decentralized control algorithm for multi-robot shepherding designed to guide a group of animals along a specified path delineated by two boundaries. The algorithm incorporates the additional objective of preserving the group within these boundaries. Simulation results reveal that, especially in sections of the path with sharp turns and a small distance between the boundaries, the group exhibits a tendency to deviate beyond the prescribed margin. Additionally, our findings emphasize the algorithm’s sensitivity to the ratio of robot-group sizes and the magnitude of the group’s velocity.

Influence of gaze, vision, and memory on hand kinematics in a placement task.

People usually reach for objects to place them in some position and orientation, but the placement component of this sequence is often ignored. For example, reaches are influenced by gaze position, visual feedback, and memory delays, but their influence on object placement is unclear. Here, we tested these factors in a task where participants placed and oriented a trapezoidal block against two-dimensional (2-D) visual templates displayed on a frontally located computer screen. In experiment 1, participants matched the block to three possible orientations: 0° (horizontal), +45° and -45°, with gaze fixated 10° to the left/right. The hand and template either remained illuminated (closed-loop), or visual feedback was removed (open-loop). Here, hand location consistently overshot the template relative to gaze, especially in the open-loop task; likewise, orientation was influenced by gaze position (depending on template orientation and visual feedback). In experiment 2, a memory delay was added, and participants sometimes performed saccades (toward, away from, or across the template). In this task, the influence of gaze on orientation vanished, but location errors were influenced by both template orientation and final gaze position. Contrary to our expectations, the previous saccade metrics also impacted placement overshoot. Overall, hand orientation was influenced by template orientation in a nonlinear fashion. These results demonstrate interactions between gaze and orientation signals in the planning and execution of hand placement and suggest different neural mechanisms for closed-loop, open-loop, and memory delay placement.NEW & NOTEWORTHY Eye-hand coordination studies usually focus on object acquisition, but placement is equally important. We investigated how gaze position influences object placement toward a 2-D template with different levels of visual feedback. Like reach, placement overestimated goal location relative to gaze and was influenced by previous saccade metrics. Gaze also modulated hand orientation, depending on template orientation and level of visual feedback. Gaze influence was feedback-dependent, with location errors having no significant effect after a memory delay.

Not seeing the forest for the trees: combination of path integration and landmark cues in human virtual navigation.

In order to successfully move from place to place, our brain often combines sensory inputs from various sources by dynamically weighting spatial cues according to their reliability and relevance for a given task. Two of the most important cues in navigation are the spatial arrangement of landmarks in the environment, and the continuous path integration of travelled distances and changes in direction. Several studies have shown that Bayesian integration of cues provides a good explanation for navigation in environments dominated by small numbers of easily identifiable landmarks. However, it remains largely unclear how cues are combined in more complex environments. To investigate how humans process and combine landmarks and path integration in complex environments, we conducted a series of triangle completion experiments in virtual reality, in which we varied the number of landmarks from an open steppe to a dense forest, thus going beyond the spatially simple environments that have been studied in the past. We analysed spatial behaviour at both the population and individual level with linear regression models and developed a computational model, based on maximum likelihood estimation (MLE), to infer the underlying combination of cues. Overall homing performance was optimal in an environment containing three landmarks arranged around the goal location. With more than three landmarks, individual differences between participants in the use of cues are striking. For some, the addition of landmarks does not worsen their performance, whereas for others it seems to impair their use of landmark information. It appears that navigation success in complex environments depends on the ability to identify the correct clearing around the goal location, suggesting that some participants may not be able to see the forest for the trees.

A comparison of hippocampal and retrosplenial cortical spatial and contextual firing patterns.

The hippocampus (HPC) and retrosplenial cortex (RSC) are key components of the brain's memory and navigation systems. Lesions of either region produce profound deficits in spatial cognition and HPC neurons exhibit well-known spatial firing patterns (place fields). Recent studies have also identified an array of navigation-related firing patterns in the RSC. However, there has been little work comparing the response properties and information coding mechanisms of these two brain regions. In the present study, we examined the firing patterns of HPC and RSC neurons in two tasks which are commonly used to study spatial cognition in rodents, open field foraging with an environmental context manipulation and continuous T-maze alternation. We found striking similarities in the kinds of spatial and contextual information encoded by these two brain regions. Neurons in both regions carried information about the rat's current spatial location, trajectories and goal locations, and both regions reliably differentiated the contexts. However, we also found several key differences. For example, information about head direction was a prominent component of RSC representations but was only weakly encoded in the HPC. The two regions also used different coding schemes, even when they encoded the same kind of information. As expected, the HPC employed a sparse coding scheme characterized by compact, high contrast place fields, and information about spatial location was the dominant component of HPC representations. RSC firing patterns were more consistent with a distributed coding scheme. Instead of compact place fields, RSC neurons exhibited broad, but reliable, spatial and directional tuning, and they typically carried information about multiple navigational variables. The observed similarities highlight the closely related functions of the HPC and RSC, whereas the differences in information types and coding schemes suggest that these two regions likely make somewhat different contributions to spatial cognition.

Convergent wheeled robot navigation based on an interpolated potential function and gradient

The article presents a novel idea to construct a smooth navigation function for a wheeled robot based on grid-based search, that enables replanning in dynamic environments. Since the dynamic constraints of the robot are also considered, the navigation function is combined with the model predictive control (MPC) to guide the robot safely to the defined goal location. The main novelty of this work is the definition of this navigation function and its MPC application with guaranteed closed-loop convergence in finite time for a non-holonomic robot with speed and acceleration constraints. The navigation function consists of an interpolated potential function derived from the grid-based search and a term that guides the orientation of the robot on continuous gradients. The navigation function guarantees convergent trajectories to the desired goal, results in smooth motion between obstacles, has no local minima, and is computationally efficient. The proposed navigation is also suitable in dynamic environments, as confirmed by experiments with a Husky mobile robot.

Frontal delta and theta power reflect strategy changes during human spatial memory retrieval in a virtual water maze task: an exploratory analysis

Brain oscillations in humans play a role in a wide range of cognitive processes, including navigation and memory. The oscillatory dynamics contributing to successful spatial memory recall in humans are not well-understood. To investigate specific oscillatory frequency bands during the recall process in human navigation, we recorded electroencephalographic (EEG) activity during a recall trial in healthy young adults (n = 15) following the learning of a goal location in a Virtual Water Maze task. We compared this to the activity during the same trial length, in a group of participants who did not learn a target location and navigated freely but were time-matched to the learning group (non-learning, n = 15). We compared relative power in Delta (2–4 Hz), Theta (5–7 Hz), Alpha (8–12 Hz), Beta (15–29 Hz), and Gamma (30–40 Hz) bands across the scalp. We found that delta and theta activity were greater during recall in our learning group, as opposed to our non-learning group. We also demonstrated clear suppression in the alpha band at posterior sites during memory-guided navigation compared to our non-learning group. Additionally, when goal-directed navigation switches to focused searching behavior, power becomes greater at the frontal region; with increases in the delta and theta bands reflecting this strategy change. There was also greater beta and gamma activity at posterior sites in our learning group. We discuss the results further in terms of the possible roles and functions of these oscillations during human navigation and hope this exploratory analysis can provide hypotheses for future spatial navigation and memory work.

Dynamic prediction of goal location by coordinated representation of prefrontal-hippocampal theta sequences

Prefrontal (PFC) and hippocampal (HPC) sequences of neuronal firing modulated by theta rhythms could represent upcoming choices during spatial memory-guided decision-making. How the PFC-HPC network dynamically coordinates theta sequences to predict specific goal locations and how it is interrupted in memory impairments induced by amyloid beta (Aβ) remain unclear. Here, we detected theta sequences of firing activities of PFC neurons and HPC place cells during goal-directed spatial memory tasks. We found that PFC ensembles exhibited predictive representation of the specific goal location since the starting phase of memory retrieval, earlier than the hippocampus. High predictive accuracy of PFC theta sequences existed during successful memory retrieval and positively correlated with memory performance. Coordinated PFC-HPC sequences showed PFC-dominant prediction of goal locations during successful memory retrieval. Furthermore, we found that theta sequences of both regions still existed under Aβ accumulation, whereas their predictive representation of goal locations was weakened with disrupted spatial representation of HPC place cells and PFC neurons. These findings highlight the essential role of coordinated PFC-HPC sequences in successful memory retrieval of a precise goal location.