Image Memorability Research Articles

Memory augmentation with an adaptive cognitive interface.

What we remember reflects both what we encounter, such as the intrinsic memorability of a stimulus, and our internal attentional state when we encounter that stimulus. Our memories are better for memorable images and images encountered in an engaged attentional state. Here, in an effort to modulate long-term memory performance, we manipulated these factors in combination by selecting the memorability of presented images contingent on individuals' natural fluctuations in sustained attention. Can image memorability and attentional state be strategically combined to improve memory? Are memorable images still well remembered during lapses in sustained attention, and conversely, can attentive states rescue memory performance for forgettable images? We designed a procedure to monitor participants' sustained attention dynamics on the fly via their response time fluctuations during a continuous performance task with trial-unique scene images. When high- or low-attentional states were detected, our algorithm triggered the presentation of high- or low-memorability images. Afterwards, participants completed a surprise recognition memory test for the attention-triggered images. Results demonstrated that memory performance for memorable items is not only resistant to lapses in sustained attention but also that memory cannot be further improved by encoding memorable items in engaged attentional states. On the other hand, memory performance for low-memorability images can be rescued by attentive encoding states. In sum, we show that both memorability and sustained attention can be leveraged in real time to maximize memory performance. This approach suggests that adaptive cognitive interfaces can tailor what information appears when to best support overall memory.

Perceiving natural images may consume less cognitive resources: evidence from image memorability, edge magnitudes, and spectral content

Perceiving natural images may consume less cognitive resources: evidence from image memorability, edge magnitudes, and spectral content

Image memorability influences memory for where the item was seen but not when.

Observers can determine whether they have previously seen hundreds of images with more than 80% accuracy. This "massive memory" for WHAT we have seen is accompanied by smaller but still massive memories for WHERE and WHEN the item was seen (spatial & temporal massive memory). Recent studies have shown that certain images are more easily remembered than others (higher "memorability"). Does memorability influence spatial massive memory and temporal massive memory? In two experiments, viewers saw 150 images presented twice in random order. These 300 images were sequentially presented at random locations in a 7 × 7 grid. If an image was categorized as old, observers clicked on the spot in the grid where they thought they had previously seen it. They also noted when they had seen it: Experiment 1-clicking on a timeline; Experiment 2-estimating the trial number when the item first appeared. Replicating prior work, data show that high-memorability images are remembered better than low-memorability images. Interestingly, in both experiments, spatial memory precision was correlated with image memorability, while temporal memory precision did not vary as a function of memorability. Apparently, properties that make images memorable help us remember WHERE but not WHEN those images were presented. The lack of correlation between memorability and temporal memory is, of course, a negative result and should be treated with caution.

Image memorability is linked to facilitated perceptual and semantic processing

Abstract Strikingly, some images are consistently more likely to be remembered compared to others—a stable, intrinsic image property that has been termed image memorability. However, the properties that afford this memory advantage have remained elusive. In prior work, we showed that more memorable images are easier to perceive, and modeling work further suggests that semantic properties contribute to variance in memorability. Thus, we hypothesize that image memorability effects arise at the interface between perception and semantic memory. To test this hypothesis, we used event-related potentials (ERPs) to measure perceptual template matching (N300) and semantic access (N400) processes in a continuous recognition memory task using high and low memorability images, each repeated once. On initial presentation, both N300 and N400 amplitudes were less negative for high memorability images, showing that memorability is linked to both facilitated high-level perceptual processing and more efficient semantic activation. High memorability images also elicited a larger N300 repetition effect compared to low memorability images, revealing that their perceptual processing benefits more from the prior exposure. The results support the idea that images that better match visual templates and elicit more targeted semantic activations are easier to identify when encountered again, and further point to a potential interplay between semantic activation and perceptual matching in supporting image memorability.

Highly memorable images are more readily perceived.

Image memorability, the likelihood that a person will remember a particular image, has been shown to be an intrinsic property of the image that is distinct from many other visual and cognitive features. Research thus far has not identified particular visual features that can sufficiently explain this intrinsic memorability, but one possibility is that more and less memorable images differ in their statistical regularity (i.e., how prototypical or distinctive they are). Statistical regularity is known to affect detection time for images, such that stimuli with higher statistical regularity can be detected with shorter presentation directions. Therefore, in the present study, we probed whether memorability affects how quickly an image can be detected. High- and low-memorability images were presented in an intact/scrambled task wherein participants were asked to indicate whether they saw an intact image or noise, and we estimated the presentation duration necessary for participants to reach 70.7% accuracy. Across two experiments using different stimulus materials, we observed and then replicated that more memorable images are associated with shorter detection thresholds than those for less memorable images. The results support the idea that memorable stimuli may better match stored templates used for image perception and/or recognition. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Light‐adaptive Mimicking Retina with In Situ Image Memorization via Resistive Switching Photomemristor Arrays

AbstractArtificial visual memory systems have been of particular interest since the development of machine vision and bionic robots. Ordinarily, the conventional system architecture requires the complex integration of two functional modules, a photo‐sensor and a memory device, which greatly limits the operating efficiency and increases the extra energy consumption. Nonetheless, other simply configured optoelectronics memory devices generally face challenges of adaption in complex light environments. Here, a resistive switching (RS) perovskite‐based photomemristor is presented that mimics the retina function. The dual function of light perception and in situ storage are both achieved. In the dark condition, it exhibits impressive memory performance with a high ON/OFF ratio of 104, a long retention time of over 104 s, and a low operating voltage of 0.38 V. With illumination, it shows self‐powered, broadband photo‐detecting characteristics with responsivity of 70 mA W−1 and detectivity of 7.5 × 1010 Jones. More importantly, benefiting from the material dual‐phase configuration, the highly steady photo‐adjusted RS windows are achieved. Its light‐adaptive memory application in dynamic environments is further demonstrated using the mimicking retina for a machine eye. This work can provide a strategy for enhanced RS photomemristor and its application in changing and varied scenarios.

Visual perception of highly memorable images is mediated by a distributed network of ventral visual regions that enable a late memorability response.

Behavioral and neuroscience studies in humans and primates have shown that memorability is an intrinsic property of an image that predicts its strength of encoding into and retrieval from memory. While previous work has independently probed when or where this memorability effect may occur in the human brain, a description of its spatiotemporal dynamics is missing. Here, we used representational similarity analysis (RSA) to combine functional magnetic resonance imaging (fMRI) with source-estimated magnetoencephalography (MEG) to simultaneously measure when and where the human cortex is sensitive to differences in image memorability. Results reveal that visual perception of High Memorable images, compared to Low Memorable images, recruits a set of regions of interest (ROIs) distributed throughout the ventral visual cortex: a late memorability response (from around 300 ms) in early visual cortex (EVC), inferior temporal cortex, lateral occipital cortex, fusiform gyrus, and banks of the superior temporal sulcus. Image memorability magnitude results are represented after high-level feature processing in visual regions and reflected in classical memory regions in the medial temporal lobe (MTL). Our results present, to our knowledge, the first unified spatiotemporal account of visual memorability effect across the human cortex, further supporting the levels-of-processing theory of perception and memory.

Predicting the memorability of scene pictures: Improved accuracy through one's own experience.

There are conflicting findings regarding the accuracy of metamemory for scene pictures. Judgements of stimulus memorability in general (memorability judgements [MJs]) have been reported to be unpredictive of actual image memorability. However, other studies have found that judgements of learning (JOLs)-predictions of one's own later memory performance for recently studied items-are moderately predictive of people's own actual recognition memory for pictures. The current study directly compared the relative accuracy and cue basis of JOLs and MJs for scene pictures. In Experiments 1 and 2, participants completed an MJ task and a JOL task in counterbalanced order. In the MJ task, they judged the general memorability of each picture. In the JOL task, they studied pictures and made JOLs during a learning phase, followed by a recognition memory test. Results showed that MJs were predictive of general scene memorability and relied on the same cues as JOLs, but MJ accuracy considerably improved after the JOL task. Experiment 3 demonstrated that prior learning experiences drove this increase in MJ accuracy. This work demonstrates that people can predict not only their own future memory performance for scene pictures with moderate accuracy but also the general memorability of scene pictures. In addition, experiences with one's own learning and memory support the ability to assess scene memorability in general. This research contributes to our understanding of the basis and accuracy of different metamemory judgements.

Brain tumor segmentation using synthetic MR images - A comparison of GANs and diffusion models

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub.

Emotional modulation of memorability in mnemonic discrimination

Although elements such as emotion may serve to enhance or impair memory for images, some images are consistently remembered or forgotten by most people, an intrinsic characteristic of images known as memorability. Memorability explains some of the variability in memory performance, however, the underlying mechanisms of memorability remain unclear. It is known that emotional valence can increase the memorability of an experience, but how these two elements interact is still unknown. Hippocampal pattern separation, a computation that orthogonalizes overlapping experiences as distinct from one another, may be a candidate mechanism underlying memorability. However, these two literatures have remained largely separate. To explore the interaction between image memorability and emotion on pattern separation, we examined performance on an emotional mnemonic discrimination task, a putative behavioral correlate of hippocampal pattern separation, by splitting stimuli into memorable and forgettable categories as determined by a convolutional neural network as well as by emotion, lure similarity, and time of testing (immediately and 24-hour delay). We measured target recognition, which is typically used to determine memorability scores, as well as lure discrimination, which taxes hippocampal pattern separation and has not yet been examined within a memorability framework. Here, we show that more memorable images were better remembered across both target recognition and lure discrimination measures. However, for target recognition, this was only true upon immediate testing, not after a 24-hour delay. For lure discrimination, we found that memorability interacts with lure similarity, but depends on the time of testing, where memorability primarily impacts high similarity lure discrimination when tested immediately but impacts low similarity lure discrimination after a 24-hour delay. Furthermore, only lure discrimination showed an interaction between emotion and memorability, in which forgettable neutral images showed better lure discrimination compared to more memorable images. These results suggest that careful consideration is required of what makes an image memorable and may depend on what aspects of the image are more memorable (e.g., gist vs. detail, emotional vs. neutral).

Children develop adult-like visual sensitivity to image memorability by the age of 4.

Adults have been shown to consistently remember and forget certain images despite large individual differences, suggesting a population-wide sensitivity to an image's intrinsic memorability-a measure of how successfully an image is remembered. While a decade of research has focused on image memorability among adults, the developmental trajectory of these consistencies in memory is understudied. Here, we investigate by what age children gain adult-like sensitivity to the image memorability effect. We utilized data from Saragosa-Harris et al. (2021), where 137 children aged between 3 and 5 years old encoded animal-scene image pairs and then after a 5-min, 24-hr, or 1-week delay performed a cued recognition task for each scene target given its animal cue. We tested adults' memory of the same scene images using ResMem (Needell & Bainbridge, 2022), a pretrained deep neural network that predicts adult image memorability scores, and using an online behavioral continuous recognition task (N = 116). Results showed that ResMem predictions, as a proxy of adults' memory, predicted scene memory of children by the age of 4 and were the most predictive of children's memory across ages after a long, 1-week delay. Children at age 3 show nonadult-like consistent memory patterns, implying that the nonadult-like memory patterns were not due to poor memory performance. Instead, 3-year-olds may have consistently used certain visual memory strategies that become less optimal as they age. Our results suggest that adult-like sensitivity to image memorability emerges by the age of 4 through experience. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Predicting memorability of face photographs with deep neural networks

With the advent of social media in our daily life, we are exposed to a plethora of images, particularly face photographs, every day. Recent behavioural studies have shown that some of these photographs stick in the mind better than others. Previous research have shown that memorability is an intrinsic property of an image, hence the memorability of an image can be computed from that image. Moreover, various works found that the memorability of an image is highly consistent across people and also over time. Recently, researchers employed deep neural networks to predict image memorability. Here, we show although those models perform well on scene and object images, they perform poorly on photographs of human faces. We demonstrate and explain why generic memorability models do not result in an acceptable performance on face photographs and propose seven different models to estimate the memorability of face images. In addition, we show that these models outperform the previous classical methods, which were used for predicting face memorability.

Memoir study: Investigating image memorability across developmental stages.

Images have been shown to consistently differ in terms of their memorability in healthy adults: some images stick in one's mind while others are forgotten quickly. Studies have suggested that memorability is an intrinsic, continuous property of a visual stimulus that can be both measured and manipulated. Memory literature suggests that important developmental changes occur throughout adolescence that have an impact on recognition memory, yet the effect that these changes have on image memorability has not yet been investigated. In the current study, we recruited adolescents ages 11-18 (n = 273, mean = 16) to an online visual memory experiment to explore the effects of developmental changes throughout adolescence on image memorability, and determine if memorability findings in adults can be generalized to the adolescent age group. We used the online experiment to calculate adolescent memorability scores for 1,000 natural images, and compared the results to the MemCat dataset-a memorability dataset that is annotated with adult memorability scores (ages 19-27). Our study finds that memorability scores in adolescents and adults are strongly and significantly correlated (Spearman's rank correlation, r = 0.76, p < 0.001). This correlation persists even when comparing adults with developmentally different sub-groups of adolescents (ages 11-14: r = 0.67, p < 0.001; ages 15-18: r = 0.60, p < 0.001). Moreover, the rankings of image categories by mean memorability scores were identical in both adolescents and adults (including the adolescent sub-groups), indicating that broadly, certain image categories are more memorable for both adolescents and adults. Interestingly, however, adolescents experienced significantly higher false alarm rates than adults, supporting studies that show increased impulsivity and reward-seeking behaviour in adolescents. Our results reveal that the memorability of images remains consistent across individuals at different stages of development. This consistency aligns with and strengthens prior research, indicating that memorability is an intrinsic property of images. Our findings open new pathways for applying memorability studies in adolescent populations, with profound implications in fields such as education, marketing, and psychology. Our work paves the way for innovative approaches in these domains, leveraging the consistent nature of image memorability across age groups.

Supplemental Material for Children Develop Adult-Like Visual Sensitivity to Image Memorability by the Age of 4

Supplemental Material for Children Develop Adult-Like Visual Sensitivity to Image Memorability by the Age of 4

Cross-cultural consistency of image memorability

Memorability refers to the intrinsic property of an image that determines how well it is remembered or forgotten. Recent studies have found that memorability is highly consistent across individuals. However, most studies on memorability were conducted with participants from Western cultures, and the images used in memorability studies were culturally biased. Previous studies implicitly assumed that memorability would be held constant across different cultural groups; however, to the best of our knowledge, this has not yet been empirically investigated. In the current study, we recruited participants from South Korea and the US and examined whether image memorability was consistent across these two cultures. We found that South Korean participants showed greater memory performance for images rated highly memorable by US participants. The current findings provide converging evidence that image memorability is not fully accounted for by individual differences, and suggest the possibility of cross-cultural consistency in image memorability.

Physical image properties influence image memorability in a category-dependent manner

Physical image properties influence image memorability in a category-dependent manner

Image memorability modulates image recognition, but not image localization in space and time

Image memorability modulates image recognition, but not image localization in space and time

What makes a visual scene more memorable? A rapid serial visual presentation (RSVP) study with dynamic visual scenes

ABSTRACT The visual system has been characterized as having limited processing capacity. Research suggests that not all visual information is equal and that certain visual scenes are registered better than others. The present study investigated how people process biological stimuli under time constraint using a Rapid Serial Visual Presentation (RSVP) paradigm with dynamic movie scenes. The results of Experiments 1 and 2 indicated that recognition memory as well as identification performance got better with longer duration (400 ms vs. 200 ms). Most importantly, biological stimuli led to better memory and lower reaction times. Lastly, Experiment 3 was conducted to replicate previously observed dynamic advantage and to disentangle the role of motion from content. The results indicated that dynamic scenes were remembered better than static scenes for both the biological and non-biological stimuli. The ecological validity and retrieval benefit of dynamic scenes were discussed in relation to image memorability.

Memory for artwork is predictable

Viewing art is often seen as a highly personal and subjective experience. However, are there universal factors that make a work of art memorable? We conducted three experiments, where we recorded online memory performance for 4,021 paintings from the Art Institute of Chicago, tested in-person memory after an unconstrained visit to the Art Institute, and obtained abstract attribute measures such as beauty and emotional valence for these pieces. Participants showed significant agreement in their memories both online and in-person, suggesting that pieces have an intrinsic "memorability" based solely on their visual properties that is predictive of memory in a naturalistic museum setting. Importantly, ResMem, a deep learning neural network designed to estimate image memorability, could significantly predict memory both online and in-person based on the images alone, and these predictions could not be explained by other low- or high-level attributes like color, content type, aesthetics, and emotion. A regression comprising ResMem and other stimulus factors could predict as much as half of the variance of in-person memory performance. Further, ResMem could predict the fame of a piece, despite having no cultural or historical knowledge. These results suggest that perceptual features of a painting play a major role in influencing its success, both in memory for a museum visit and in cultural memory over generations.

Memorable beginnings, but forgettable endings: Intrinsic memorability alters our subjective experience of time

ABSTRACT Time is the fabric of experience – yet it is incredibly malleable in the mind of the observer: seeming to drag on, or fly right by at different moments. One of the most influential drivers of temporal distortions is attention, where heightened attention dilates subjective time. But an equally important feature of subjective experience involves not just the objects of attention, but also what information will naturally be remembered or forgotten, independent of attention (i.e., intrinsic image memorability). Here we test how memorability influences time perception. Observers viewed scenes in an oddball paradigm, where the last scene could be a forgettable “oddball” amidst memorable ones, or vice versa. Subjective time dilation occurred only for forgettable oddballs, but not memorable ones – demonstrating an oddball effect where the oddball did not differ in low-level visual features, image category, or even subjective memorability. But more importantly, these results emphasize how memory can interact with temporal experience: memorable beginnings may put people in an efficient encoding state, which may in turn influence which moments are dilated in time.