Task-critical Information Research Articles

A Model-Agnostic Graph Neural Network for Integrating Local and Global Information

Graph Neural Networks (GNNs) have achieved promising performance in a variety of graph-focused tasks. Despite their success, however, existing GNNs suffer from two significant limitations: a lack of interpretability in their results due to their black-box nature, and an inability to learn representations of varying orders. To tackle these issues, we propose a novel Model-agnostic Graph Neural Network (MaGNet) framework, which is able to effectively integrate information of various orders, extract knowledge from high-order neighbors, and provide meaningful and interpretable results by identifying influential compact graph structures. In particular, MaGNet consists of two components: an estimation model for the latent representation of complex relationships under graph topology, and an interpretation model that identifies influential nodes, edges, and node features. Theoretically, we establish the generalization error bound for MaGNet via empirical Rademacher complexity, and demonstrate its power to represent layer-wise neighborhood mixing. We conduct comprehensive numerical studies using simulated data to demonstrate the superior performance of MaGNet in comparison to several state-of-the-art alternatives. Furthermore, we apply MaGNet to a real-world case study aimed at extracting task-critical information from brain activity data, thereby highlighting its effectiveness in advancing scientific research.

Hippocampal ensembles represent sequential relationships among an extended sequence of nonspatial events

The hippocampus is critical to the temporal organization of our experiences. Although this fundamental capacity is conserved across modalities and species, its underlying neuronal mechanisms remain unclear. Here we recorded hippocampal activity as rats remembered an extended sequence of nonspatial events unfolding over several seconds, as in daily life episodes in humans. We then developed statistical machine learning methods to analyze the ensemble activity and discovered forms of sequential organization and coding important for order memory judgments. Specifically, we found that hippocampal ensembles provide significant temporal coding throughout nonspatial event sequences, differentiate distinct types of task-critical information sequentially within events, and exhibit theta-associated reactivation of the sequential relationships among events. We also demonstrate that nonspatial event representations are sequentially organized within individual theta cycles and precess across successive cycles. These findings suggest a fundamental function of the hippocampal network is to encode, preserve, and predict the sequential order of experiences.

Investigating a Virtual Reality-based Emergency Response Scenario and Intelligent User Interface for First Responders

Because advances in broadband capabilities will soon allow first responders to access and use many forms of data when responding to emergencies, it is becoming critically important to design heads-up displays to present first responders with information in a manner that does not induce extraneous mental workload or cause undue interaction errors. Virtual reality offers a unique medium for envisioning and testing user interface concepts in a realistic and controlled environment. In this paper, we describe a virtual reality-based emergency response scenario that was designed to support user experience research for evaluating the efficacy of intelligent user interfaces for firefighters. We describe the results of a usability test that captured firefighters’ feedback and reactions to the VR scenario and the prototype intelligent user interface that presented them with task critical information through the VR headset. The paper concludes with lessons learned from our development process and a discussion of plans for future research.

Near-Optimal Reactive Synthesis Incorporating Runtime Information.

We consider the problem of optimal reactive synthesis - compute a strategy that satisfies a mission specification in a dynamic environment, and optimizes a performance metric. We incorporate task-critical information, that is only available at runtime, into the strategy synthesis in order to improve performance. Existing approaches to utilising such time-varying information require online re-synthesis, which is not computationally feasible in real-time applications. In this paper, we pre-synthesize a set of strategies corresponding to candidate instantiations (pre-specified representative information scenarios). We then propose a novel switching mechanism to dynamically switch between the strategies at runtime while guaranteeing all safety and liveness goals are met. We also characterize bounds on the performance suboptimality. We demonstrate our approach on two examples - robotic motion planning where the likelihood of the position of the robot's goal is updated in real-time, and an air traffic management problem for urban air mobility.

CAMEL: An Adaptive Camera With Embedded Machine Learning-Based Sensor Parameter Control

Today’s cameras are designed to approximate what they observe in a manner that preserves entropy. However, time critical autonomous applications such as autonomous driving, surveillance and defense systems require task critical information at the highest quality. With rapid advances in frame rates and resolutions, observing scenes at the highest quality raises concerns for the transmission bandwidth. In this paper, we introduce a new paradigm of smart camera that captures only task-critical information at the highest quality. Embedded deep neural network (DNN) algorithms within the camera enhance quality of information through real-time control of sensor parameters. We show the hardware feasibility of this camera by demonstrating a 3D-stacked architecture with a Digital Pixel Sensor (DPS). We demonstrate a number of high-level vision applications that benefit through this task-guided control including object detection, object tracking and activity recognition. Finally, we present the unique challenges faced created as a result of feedback and show how software/hardware innovations can be used to mitigate them.

Fluid Intelligence Predicts Novel Rule Implementation in a Distributed Frontoparietal Control Network.

Fluid intelligence has been associated with a distributed cognitive control or multiple-demand (MD) network, comprising regions of lateral frontal, insular, dorsomedial frontal, and parietal cortex. Human fluid intelligence is also intimately linked to task complexity, and the process of solving complex problems in a sequence of simpler, more focused parts. Here, a complex target detection task included multiple independent rules, applied one at a time in successive task epochs. Although only one rule was applied at a time, increasing task complexity (i.e., the number of rules) impaired performance in participants of lower fluid intelligence. Accompanying this loss of performance was reduced response to rule-critical events across the distributed MD network. The results link fluid intelligence and MD function to a process of attentional focus on the successive parts of complex behavior.SIGNIFICANCE STATEMENT Fluid intelligence is intimately linked to the ability to structure complex problems in a sequence of simpler, more focused parts. We examine the basis for this link in the functions of a distributed frontoparietal or multiple-demand (MD) network. With increased task complexity, participants of lower fluid intelligence showed reduced responses to task-critical events. Reduced responses in the MD system were accompanied by impaired behavioral performance. Low fluid intelligence is linked to poor foregrounding of task-critical information across a distributed MD system.

A Scalar Neural Code for Categories in Parietal Cortex: Representing Cognitive Variables as “More” or “Less”

In this issue, Fitzgerald et al. (2013) show that LIP neurons in monkeys encode categorically distinct task conditions using a scalar code. Activity scales up or down to encode different categories, with neurons maintaining proportional levels of activity in relation to one another.

Adaptive Coding of Task-Relevant Information in Human Frontoparietal Cortex

Frontoparietal cortex is thought to be essential for flexible behavior, but the mechanism for control remains elusive. Here, we demonstrate a potentially critical property of this cortex: its dynamic configuration for coding of task-critical information. Using multivoxel pattern analysis of human functional imaging data, we demonstrate an adaptive change in the patterns of activation coding task-relevant stimulus distinctions. When task demands made perceptual information more difficult to discriminate, frontoparietal regions showed increased coding of this information. Visual cortices showed the opposite result: a weaker representation of perceptual information in line with the physical change in the stimulus. On a longer timescale, a rebalancing of coding was also seen after practice, with a diminished representation of task rules as they became familiar. The results suggest a flexible neural system, exerting cognitive control in a wide range of tasks by adaptively representing the task features most challenging for successful goal-directed behavior.

Visual Search Asymmetries in Heavy Clutter: Implications for Display Design

An experiment aimed to test whether design of symbology to produce visual search asymmetries might facilitate target detection in cluttered displays. A visual search asymmetry exists between two stimuli when a target of one type is found efficiently among distractors of the second type but a target of the second type is found with difficulty among distractors of the first type. Asymmetries have generally been studied within relatively sparse displays. In the present study, the authors tested whether an asymmetry driven by stimulus familiarity persists within heavily cluttered imagery. In this study, 10 participants performed a visual search task using stimuli (canonical vs. reversed Ns) known to produce a search asymmetry. Search stimuli were embedded within geospatial images containing either low or high levels of clutter. A decision theoretic index of sensitivity served as the dependent measure. The search asymmetry was robust against the presence of heavy display clutter. Specifically, sensitivity was greater when the target was a reversed N rather than an N, and this pattern remained within cluttered displays. Time-accuracy analysis revealed that the search asymmetry increased the rate of information accumulation roughly equally within low- and high- clutter images. Search asymmetries are robust against heavy, spatially continuous visual clutter. Design of display symbology to produce visual search asymmetries can offset the costs of visual clutter, maximizing detectability of task-critical information in complex displays.

Affective Biases in Information Search during Tactical Decision-Making

Tactical decision-making may be an emotional experience for an operator. Emotion, whether extrinsic or intrinsic to the task, may bias attention to task critical information. The current study used a feedback manipulation to induce positive and negative affect (PA and NA) during a decision-making task requiring information search. In a task based on a rescue scenario, participants were randomly assigned to either a “success” or “failure” condition and instructed to evaluate the costs and benefits of different routes along separate legs of the rescue mission, similar to reaching a “fork in the road.” Feedback was effective in manipulating emotion. PA and NA were significantly correlated with information sampling frequencies, but no general mood-congruent bias was found. Instead the role of mood appears to depend on the general affective context, a finding interpreted in relation to the mood-as-input hypothesis. The practical relevance and limitations of the study are discussed.

Visual search asymmetries within color-coded and intensity-coded displays.

Color and intensity coding provide perceptual cues to segregate categories of objects within a visual display, allowing operators to search more efficiently for needed information. Even within a perceptually distinct subset of display elements, however, it may often be useful to prioritize items representing urgent or task-critical information. The design of symbology to produce search asymmetries (Treisman & Souther, 1985) offers a potential technique for doing this, but it is not obvious from existing models of search that an asymmetry observed in the absence of extraneous visual stimuli will persist within a complex color- or intensity-coded display. To address this issue, in the current study we measured the strength of a visual search asymmetry within displays containing color- or intensity-coded extraneous items. The asymmetry persisted strongly in the presence of extraneous items that were drawn in a different color (Experiment 1) or a lower contrast (Experiment 2) than the search-relevant items, with the targets favored by the search asymmetry producing highly efficient search. The asymmetry was attenuated but not eliminated when extraneous items were drawn in a higher contrast than search-relevant items (Experiment 3). Results imply that the coding of symbology to exploit visual search asymmetries can facilitate visual search for high-priority items even within color- or intensity-coded displays.

A Signal Detection Analysis of the Effects of Workload, Task-Critical and Likelihood Information on Human Alarm Response

The purpose of this study was to examine the effects of task-critical and likelihood information on participants' sensitivity and bias to alarm signals under varying levels of workload. Participants performed a complex primary task at the same time they performed a secondary task. Likelihood information was manipulated through the use of either a Binary Alarm System (BAS) or a Likelihood Alarm System (LAS). As expected, task-critical and likelihood information significantly increased participants' sensitivity, and this varied across workload levels. Participants benefited from task-critical information only when they were interacting with the BAS. However, participants benefited from likelihood information regardless of task-critical information, particularly under high-workload conditions. Furthermore, task-critical information increased participants' response bias under low workload, making them less likely to respond to alarm signals. These results demonstrated the superior advantage of an LAS over a traditional BAS and showed support for the use of an LAS as a way to mitigate the cry-wolf effect above and beyond task-critical information.

Combating Cry Wolf: The Effects of Synthetic Verbal Alarm Urgency on Choice Reaction Performance

Researchers have shown that increasing the perceived urgency of nonverbal auditory alarms can meliorate alarm response performance degradation due to the cry wolf effect. We conducted two experiments to examine the effects of verbal alarm urgency on the cry wolf effect. Experiment 1 revealed that participants responded significantly less often and correctly rejected more urgent alarms. Experiment 2 revealed that the results in Experiment 1 were contingent on the presence of additional task critical information. Response degradation due to the cry wolf effect was not moderated by alarm urgency in either study. However, the results suggest that performance may still benefit from increased alarm urgency provided additional task critical information is available. These results can be utilized to design verbal alarm systems that evoke more appropriate reactions to false alarms.

Cortical involvement in acquisition and extinction of trace eyeblink conditioning.

Previous studies have implicated 2 cortical regions interconnected with the hippocampal formation, the retrosplenial cortex (RSC) and the medial prefrontal cortex (mPFC), as loci important for the acquisition of hippocampally dependent trace eyeblink conditioning. These loci have also been proposed to serve as long-term storage sites of task critical information. This study used lesions made prior to training to investigate the roles of the RSC, as well as the caudal and rostral subdivisions of the mPFC, in the acquisition and subsequent extinction of trace eyeblink conditioning in the rabbit. The caudal mPFC and rostral mPFC were shown to be critical for acquisition and extinction of the conditioned reflex, respectively. The data indicate that the RSC is not critical for acquisition or extinction of the trace conditioned reflex.