Subsequent Recognition Task Research Articles

The effects of experimental pain on episodic memory and its top-down modulation: a preregistered pooled analysis.

Pain can automatically interfere with ongoing cognitive processes such as attention and memory. The extent of pain's negative effects on cognitive functioning seems to depend on a balance between top-down and bottom-up factors. In this large, preregistered, pooled reanalysis of 8 studies, we investigated the robustness of the detrimental effect of acute pain on recognition memory and whether top-down mechanisms such as pain-related expectations or cognitions (pain-related fear, pain catastrophizing) modulate this effect. Two hundred forty-seven healthy participants underwent similar experimental paradigms, including a visual categorization task with images randomly paired with (or without) concomitant painful stimulation and a subsequent unannounced recognition task. Recognition memory (ie, d', recollection, and familiarity) and categorization performance (ie, reaction time, accuracy) served as proxies for the effect of pain on cognitive performance. Acute painful stimulation significantly impaired recognition performance (d', familiarity). However, recognition performance was not significantly modulated by participants' expectations regarding the effect of pain on task performance or pain-related cognitions in this sample of healthy participants. Our results corroborate the negative effects of pain on (visual) memory encoding reported in previous studies and reports of "memory problems" from patients with chronic pain. To characterize the role of bottom-up and top-down factors for the detrimental effects of pain, large-scale studies with more nuanced study designs are necessary. Future studies in patient cohorts must unravel the interaction of maladaptive pain-related cognitions and the often-reported impaired cognitive performance in chronic pain patients.

Image Deraining Algorithm Based on Multi-Scale Features

In target detection, tracking, and recognition tasks, high-quality images can achieve better results. However, in actual scenarios, the visual effects and data quality of images are greatly reduced due to the influence of environmental factors, which affect subsequent detection, recognition, and other tasks. Therefore, this paper proposes an image rain removal algorithm based on multi-scale features, which can effectively remove rain streaks. First of all, this paper proposes a deraining algorithm that combines spatial information to improve the network’s generalization ability on real images, aiming at the problem of synthetic datasets used by previous deraining algorithms. Then, by proposing a multi-scale rain removal algorithm, it improves the feature extraction capabilities of existing deraining algorithms. Before extracting deep rain features, a preliminary fusion of multi-scale shallow features can be performed, which can show better performance in images of different sizes. In addition, a spatial attention module and channel are introduced. The attention module increases the ability to extract rain information at each scale; the resulting multi-scale feature image rain removal algorithm is called MFD. Finally, the rain removal algorithm is validated on the rain removal dataset, and the proposed method can effectively remove rain patterns, provide strong performance improvement on several datasets in the image rain removal task, and provide high-quality images for subsequent detection and recognition tasks.

Thermal image visualization using multi-discriminator CycleGAN with unpaired thermal–visible image training set

Thermal imaging utilizes the differences in thermal radiation within a scene to capture images, effectively compensating for the limited imaging capabilities of visible-light cameras in low-light environments. In order to further enhance thermal imaging, we propose a novel approach that transforms the visualization of thermal images into an image conversion task. Our method, named TIVNet (Thermal Images Visualization Network), is based on a multi-discriminator cycle-consistent adversarial network (CycleGAN) and operates in the YUV color space to accurately colorize and represent thermal infrared images. The utilization of color and texture discriminators in our framework ensures that the converted images exhibit appropriate color styles and capture fine details, respectively. In the generator, we employed a Dense Residual Module (DRM) to facilitate the reuse of features and enhance the flow of information. By utilizing the DRM, we aim to enhance the overall performance and efficiency of the generator network. Additionally, we incorporated the Dual-channel Self-Attention Module(DCSA) to enhance the feature representation. By leveraging self-attention, the generator can effectively learn and exploit the inherent relationships between different regions, leading to improved feature representation and, consequently, more accurate colorization and representation of thermal infrared images. Notably, our proposed method leverages unsupervised learning, eliminating the need for a fully paired thermal infrared-visible image dataset. Through extensive experimentation on publicly available datasets as well as our own, we demonstrate that our approach outperforms state-of-the-art methods in thermal image visualization. Furthermore, the converted images produced by TIVNet exhibit a high level of realism, thereby enhancing the visual impact of thermal imaging. This improvement is particularly beneficial for human perception and subsequent target recognition tasks. Our findings suggest that the application of our method yields superior results both quantitatively and qualitatively, showcasing its potential for advancing thermal imaging capabilities.

Increases in pre-stimulus theta and alpha oscillations precede successful encoding of crossmodal associations

A central aspect of episodic memory is the formation of associations between stimuli from different modalities. Current theoretical approaches assume a functional role of ongoing oscillatory power and phase in the theta band (3–7 Hz) for the encoding of crossmodal associations. Furthermore, ongoing activity in the theta range as well as alpha (8–12 Hz) and low beta activity (13–20 Hz) before the presentation of a stimulus is thought to modulate subsequent cognitive processing, including processes that are related to memory. In this study, we tested the hypothesis that pre-stimulus characteristics of low frequency activity are relevant for the successful formation of crossmodal memory. The experimental design that was used specifically allowed for the investigation of associative memory independent from individual item memory. Participants (n = 51) were required to memorize associations between audiovisual stimulus pairs and distinguish them from newly arranged ones consisting of the same single stimuli in the subsequent recognition task. Our results show significant differences in the state of pre-stimulus theta and alpha power between remembered and not remembered crossmodal associations, clearly relating increased power to successful recognition. These differences were positively correlated with memory performance, suggesting functional relevance for behavioral measures of associative memory. Further analysis revealed similar effects in the low beta frequency ranges, indicating the involvement of different pre-stimulus-related cognitive processes. Phase-based connectivity measures in the theta band did not differ between remembered and not remembered stimulus pairs. The findings support the assumed functional relevance of theta band oscillations for the formation of associative memory and demonstrate that an increase of theta as well as alpha band oscillations in the pre-stimulus period is beneficial for the establishment of crossmodal memory.

Long-term memory for distractors: Effects of involuntary attention from working memory.

In a visual search task, attention to task-irrelevant distractors impedes search performance. However, is it maladaptive to future performance? Here, I showed that attended distractors in a visual search task were better remembered in long-term memory (LTM) in the subsequent surprise recognition task than non-attended distractors. In four experiments, participants performed a visual search task using real-world objects of a single color. They encoded color in working memory (WM) during the task; because each object had a different color, participants directed their attention to the WM-matching colored distractor. Then, in the surprise recognition task, participants were required to indicate whether an object had been shown in the earlier visual search task, regardless of its color. The results showed that attended distractors were remembered better in LTM than non-attended distractors (Experiments 1 and 2). Moreover, the more participants directed their attention to distractors, the better they explicitly remembered them. Participants did not explicitly remember the color of the attended distractors (Experiment 3) but remembered integrated information with object and color (Experiment 4). When the color of the distractors in the recognition task was mismatched with the color in the visual search task, LTM decreased compared to color-matching distractors. These results suggest that attention to distractors impairs search for a target but is helpful in remembering distractors in LTM. When task-irrelevant distractors become task-relevant information in the future, their attention becomes beneficial.

Neurocognitive predictors of food memory in healthy adults – A preregistered analysis

Memory processes have long been known to determine food choices (Rozin & Zellner, 1985) but recognition memory of food and its cognitive, homeostatic and neuroanatomical predictors are still largely understudied.60 healthy, overweight, non-restrictive eating adults (20 females) took part in a food wanting and subsequent food recognition and lure discrimination task at four time points after a standardized breakfast shake. With advanced tractography of 3 T diffusion-weighted imaging data, we investigated the influence of the uncinate fasciculus’ (UF) brain microstructure on the interplay of food wanting and memory processes. The analysis was preregistered in detail and conducted with Bayesian multilevel regression modeling.Target recognition (d’) and lure discrimination (LDI) performance of food tended to be higher than of art images while single image food memory accuracy evidently dominated art memory. On this single item level, wanting enhanced recognition accuracy and caloric content enhanced food memory accuracy. The enhancement by reward anticipation was most pronounced during memory encoding. Subjective hunger level did not predict performance on the memory task. The microstructure of the UF did neither evidently affect memory performance outcomes nor moderate the wanting enhancement of the recognition accuracy. Interestingly, female participants outperformed males on the memory task, and individuals with stronger neuroticism showed poorer memory performance.We shed light on to date understudied processes in food decision-making: reward anticipation influenced recognition accuracy and food memory was enhanced by higher caloric content, both effects might shape food decisions. Our findings indicate that brain microstructure does not affect food decision processes in adult populations with overweight. We suggest extending investigation of this interplay to brain activity as well as to populations with eating behaviour disorders.

Human Action Recognition Using Key-Frame Attention-Based LSTM Networks

Human action recognition is a classical problem in computer vision and machine learning, and the task of effectively and efficiently recognising human actions is a concern for researchers. In this paper, we propose a key-frame-based approach to human action recognition. First, we designed a key-frame attention-based LSTM network (KF-LSTM) using the attention mechanism, which can be combined with LSTM to effectively recognise human action sequences by assigning different weight scale values to give more attention to key frames. In addition, we designed a new key-frame extraction method by combining an automatic segmentation model based on the autoregressive moving average (ARMA) algorithm and the K-means clustering algorithm. This method effectively avoids the possibility of inter-frame confusion in the temporal sequence of key frames of different actions and ensures that the subsequent human action recognition task proceeds smoothly. The dataset used in the experiments was acquired with an IMU sensor-based motion capture device, and we separately extracted the motion features of each joint using a manual method and then performed collective inference.

An ERP study on novel word learning in an immersive virtual reality context

AbstractIn this behavioral and electrophysiological study, we compare novel word learning, particularly lexical form acquisition, in an immersive virtual reality (VR) context with a picture-word (PW) association context. We also test whether inhibitory control and age of second language acquisition (L2 AoA) have modulating effects. Chinese speakers of L2 English learned two sets of German words, one set in each of the contexts. Behavioral performance from a subsequent recognition task indicated that responses to VR-learned words were faster than PW-leaned words. ERPs revealed that VR-learned words elicited more negative N100 and N200 waveforms than PW-learned words. Moreover, a significant relationship between L2 AoA and N200 amplitude was observed for VR-learned words. Taken together, the results suggest that the multi-sensory, interactive experience simulated by an immersive VR context has a positive effect on early lexical form acquisition of novel words.

"What Did I Tell This Sad Person?": Memory for Emotional Destinations in Korsakoff's Syndrome.

We investigated destination memory, defined as the ability to remember to whom a piece of information was previously transmitted, for emotional destinations (i.e., a happy or sad person) in Korsakoff's syndrome (KS). We asked patients with KS and control participants to tell facts to neutral, positive, or negative faces. On a subsequent recognition task, participants had to decide to whom they told each fact. Compared with control participants, patients with KS demonstrated lower recognition of neutral, emotionally positive, and emotionally negative destinations. Patients with KS demonstrated lower recognition of emotionally negative than for emotionally positive or neutral destinations, but there were no significant differences between recognition of neutral and emotionally positive destinations. Our study demonstrates a compromised ability to process negative destinations in KS. Our study highlights the relationship between memory decline and impaired emotional processing in KS.

Relation between event segmentation and memory dysfunction in Parkinson’s disease

The perception of everyday events is thought to imply the segmentation into discrete sub-events. Involvement of dopaminergic networks in this process could relate to particular problems of persons with Parkinson’s disease (PD) to recall recent activities. In an event segmentation task, persons with PD and healthy controls had to indicate the beginning of sub-events within three movies showing persons performing everyday activities. In a subsequent recognition task, they should judge whether presented pictures of sub-events were part of the watched movies. In a final order memory task, they had to arrange pictures in the sequence in which they had occurred. With respect to the overall segmentation behavior, persons with PD diverged from healthy controls only in the most familiar of the three demonstrated everyday activities. Moreover, persons with PD compared to healthy controls showed generally worse event recognition and committed more errors in the order memory task. These memory deficits were the higher, the more the segmentation moved away from the ‘normative’ segmentation pattern identified in healthy controls. The findings suggest that dysfunctional structuring of sensory event information contributes to deficient event representations of ongoing everyday activities and recall problems of these recently perceived events in persons with PD.

Recruitment of a long-term memory supporting neural network during repeated maintenance of a multi-item abstract visual image in working memory

Humans can flexibly transfer information between different memory systems. Information in visual working memory (VWM) can for instance be stored in long-term memory (LTM). Conversely, information can be retrieved from LTM and temporarily held in WM when needed. It has previously been suggested that a neural transition from parietal- to midfrontal activity during repeated visual search reflects transfer of information from WM to LTM. Whether this neural transition indeed reflects consolidation and is also observed when memorizing a rich visual scene (rather than responding to a single target), is not known. To investigate this, we employed an EEG paradigm, in which abstract six-item colour-arrays were repeatedly memorized and explicitly visualized, or merely attended to. Importantly, we tested the functional significance of a potential neural shift for longer-term consolidation in a subsequent recognition task. Our results show a gradually enhanced- and sustained modulation of the midfrontal P170 component and a decline in parietal CDA, during repeated WM maintenance. Improved recollection/visualization of memoranda upon WM-cueing, was associated with contralateral parietal- and right temporal activity. Importantly, only colour-arrays previously held in WM, induced a greater midfrontal P170-response, together with left temporal- and late centro-parietal activity, upon re-exposure. These findings provide evidence for recruitment of an LTM-supporting neural network which facilitates visual WM maintenance.

Zero-Shot Learning Based on Multitask Extended Attribute Groups

Since the learning of attribute classifiers is independent of the learning of object classifier in zero-shot learning, it is difficult to guarantee that the learned attribute classifiers are optimal for the subsequent object recognition tasks. Therefore, a novel zero-shot learning method based on multitask extended attribute groups (MTEAGs) is proposed by using the multitask learning framework and grouping idea. First, we used an unsupervised clustering method to group the attributes and object classes of training images. Then, based on the obtained attribute and class groups, we constructed the group-based attribute/object classifier collaborative learning model where the class groups are viewed as the extension of attribute groups. In order to explore the shared features within a group as well restrict the feature sharing between groups, we applied the structured sparse method to constrain the model parameter matrix. At last, a hybrid zero-shot classifying model is designed by simultaneously considering the class–class and class–attribute matrices to predict the class labels of testing images, where the class–class relationship is measured by the Jaccard similarity coefficient. Experiments on two popular attribute datasets show that, MTEAG can yield higher zero-shot image classification accuracy compared with several baselines.

A Functional MRI Paradigm for Efficient Mapping of Memory Encoding Across Sensory Conditions.

We introduce a new and time-efficient memory-encoding paradigm for functional magnetic resonance imaging (fMRI). This paradigm is optimized for mapping multiple contrasts using a mixed design, using auditory (environmental/vocal) and visual (scene/face) stimuli. We demonstrate that the paradigm evokes robust neuronal activity in typical sensory and memory networks. We were able to detect auditory and visual sensory-specific encoding activities in auditory and visual cortices. Also, we detected stimulus-selective activation in environmental-, voice-, scene-, and face-selective brain regions (parahippocampal place and fusiform face area). A subsequent recognition task allowed the detection of sensory-specific encoding success activity (ESA) in both auditory and visual cortices, as well as sensory-unspecific positive ESA in the hippocampus. Further, sensory-unspecific negative ESA was observed in the precuneus. Among others, the parallel mixed design enabled sustained and transient activity comparison in contrast to rest blocks. Sustained and transient activations showed great overlap in most sensory brain regions, whereas several regions, typically associated with the default-mode network, showed transient rather than sustained deactivation. We also show that the use of a parallel mixed model had relatively little influence on positive or negative ESA. Together, these results demonstrate a feasible, versatile, and brief memory-encoding task, which includes multiple sensory stimuli to guarantee a comprehensive measurement. This task is especially suitable for large-scale clinical or population studies, which aim to test task-evoked sensory-specific and sensory-unspecific memory-encoding performance as well as broad sensory activity across the life span within a very limited time frame.

DetectGAN: GAN-based text detector for camera-captured document images

Nowadays, with the development of electronic devices, more and more attention has been paid to camera-based text processing. Different from scene image, the recognition system of document image needs to sort out the recognition results and store them in the structured document for the subsequent data processing. However, in document images, the fusion of text lines largely depends on their semantic information rather than just the distance between the characters, which causes the problem of learning confusion in training. At the same time, for multi-directional printed characters in document images, it is necessary to use additional directional information to guide subsequent recognition tasks. In order to avoid learning confusion and get recognition-friendly detection results, we propose a character-level text detection framework, DetectGAN, based on the conditional generative adversarial networks (abbreviation cGAN used in the text). In the proposed framework, position regression and NMS process are removed, and the problem of text detection is directly transformed into an image-to-image generation problem. Experimental results show that our method has an excellent effect on text detection of camera-captured document images and outperforms the classical and state-of-the-art algorithms.

Discriminative sparsity preserving projection via global constraint for unconstrained face recognition

The unconstrained face images collected in the real environments include many complicated and changeable interference factors, and sparsity preserving projections cannot well characterise the low-dimensional intrinsic structure embedded in the high-dimensional unconstrained face images, which is important for subsequent recognition task. To deal with this problem, in this study the authors propose a new dimensionality reduction method named as discriminative sparsity preserving projection via global constraint. It seeks an optimal sub-space in which the samples in intra-classes are as compact as possible, while the samples in inter-classes are as separable as possible by adopting the compactness constraint terms of reconstruction coefficients and the penalty terms of global distribution. Extensive experiments are carried out on Faces in Labeled the Wild database and PubFig database which are two representative unconstrained face sets, and the corresponding experimental results illustrate the effectiveness of the proposed method .

A Scene Text Detector for Text with Arbitrary Shapes

The performance of text detection is crucial for the subsequent recognition task. Currently, the accuracy of the text detector still needs further improvement, particularly those with irregular shapes in a complex environment. We propose a pixel-wise method based on instance segmentation for scene text detection. Specifically, a text instance is split into five components: a Text Skeleton and four Directional Pixel Regions, then restoring itself based on these elements and receiving supplementary information from other areas when one fails. Besides, a Confidence Scoring Mechanism is designed to filter characters similar to text instances. Experiments on several challenging benchmarks demonstrate that our method achieves state-of-the-art results in scene text detection with an F-measure of 84.6% on Total-Text and 86.3% on CTW1500.

In Medio Stat Virtus: intermediate levels of mind wandering improve episodic memory encoding in a virtual environment.

Episodic memory encoding is highly influenced by the availability of attentional resources. Mind wandering corresponds to a shift of attention toward task-unrelated thoughts. Few studies, however, have tested this link between memory encoding and mind wandering. The goal of the present work was to systematically investigate the influence of mind wandering during encoding on episodic memory performances in an ecological setting. Fifty-two participants were asked to navigate in a virtual urban environment. During the walk, they encountered different scenes that, unbeknownst to the participants, were target items presented in a subsequent recognition task associated with a Remember-Know-Guess paradigm. Each item triggered, after a random interval, a thought probe assessing current mind wandering. We found a significant linear positive relationship between the ratio of correctly recognized items and the overall mind wandering reported after the task. Moreover, we found a quadratic reversed U-shaped relationship between the probability of giving a 'Remember' response and both on-line and mind wandering reported a posteriori. The nearer to the medium value the level of mind wandering was, the higher was the probability to have a recollection-based recognition. Our results indicate that in a complex environment, the highest probability of actually remembering a scene would be when participants present a medium attentional level: neither distracted by inner thoughts nor too focused on the environment. This open attentional state would allow a better global processing of the environment by preventing one's attention from being captured by internal thoughts or narrowed by an over-focusing on the environment.

A rapid, neural measure of implicit recognition memory using fast periodic visual stimulation

Fast periodic visual stimulation (FPVS) has recently emerged as a powerful new tool in cognitive neuroscience. Capable of measuring a range of cognitive functions in single subjects in just minutes of recording time, it has been adapted to measure visual, semantic and linguistic processing. We present a new adaptation of the FPVS approach to measure recognition memory via old/new contrasts. Twenty one subjects (23 (±6) yrs, 7 males) completed an FPVS-oddball paradigm that assessed their spontaneous ability to differentiate between rapidly presented images on the basis of a pre-FPVS encoding task, i.e. oddball stimuli were only defined by the subject’s experimentally induced memory of them. A clear oddball detection response reflecting recognition memory was observed within one minute of EEG recording time, simply through the passive viewing of stimuli, i.e. subjects received no task instructions and provided no behavioural response. Performance on a subsequent behavioural recognition task showed high levels of recognition of the oddball stimuli. As such, the FPVS approach returned an objective, non-verbal measure of recognition memory in just one minute of recording time, free from the confounds of behavioural recognition tasks. This finding reinforces the adaptability of the FPVS approach for the examination of higher-level cognition and provides a new method for the neural measurement of recognition memory.

Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression

In real video surveillance scenes, the extracted face regions generally have low-resolution (LR) and are sensitive to pose and illumination variations; these flaws undoubtedly degrade the subsequent recognition task. To overcome these challenges, we propose an approach named multilayer locality-constrained structural orthogonal Procrustes regression (MLCSOPR). The proposed MLCSOPR not only learns the pose-robust discriminative representation features to reduce the resolution gap between the LR image space and the high-resolution (HR) one but also strengthens the consistency between the LR and HR image space. In particular, several contributions are made in this paper: (i) Inspired by the orthogonal Procrustes problem (OPP), a matrix approximation is exploited to find an optimal correction between two data matrices. (ii) The nuclear norm constraint is applied to the reconstruction error to maintain the structural property. (iii) Based on the abovementioned learned resolution-robust representation features, a linear regression-based classification strategy is adopted to recognize the LR input face images. Experiments on commonly used face databases have shown the effectiveness of the proposed method on cross-resolution face matching with pose variations.

Structure-function associations of successful associative encoding

Functional magnetic resonance imaging (MRI) studies have demonstrated a critical role of hippocampus and inferior frontal gyrus (IFG) in associative memory. Similarly, evidence from structural MRI studies suggests a relationship between gray-matter volume in these regions and associative memory. However, how brain volume and activity relate to each other during associative-memory formation remains unclear. Here, we used joint independent component analysis (jICA) to examine how gray-matter volume and brain activity would be associated during associative encoding, especially in medial-temporal lobe (MTL) and IFG. T1-weighted images were collected from 27 young adults, and functional MRI was employed during intentional encoding of object pairs. A subsequent recognition task tested participants’ memory performance. Unimodal analyses using voxel-based morphometry revealed that participants with better associative memory showed larger gray-matter volume in left anterior hippocampus. Results from the jICA revealed one component that comprised a covariance pattern between gray-matter volume in anterior and posterior MTL and encoding-related activity in IFG. Our findings suggest that gray matter within the MTL modulates distally distinct parts of the associative encoding circuit, and extend previous studies that demonstrated MTL-IFG functional connectivity during associative memory tasks.