Memory Chunks Research Articles

How Time Pressure Modulates Individual Differences in the Functional Connectivity of Chunk Memory in Chess Games

Previous studies on the chess game demonstrated that chess experts strongly rely on the activation of memory chunks to manifest accurate decision-making. Although the chunk memory might be affected by temporal constraints, it is unclear why the performance of chess experts is not significantly dropped under time pressure. In this study, our objective is to examine the variations in cognitive neural mechanisms between chess experts and novices under time pressure. The underlying cognitive neural mechanism was carefully inspected by accessing the chess game performance between 20 local experienced and 20 inexperienced chess players with 1-minute and 5-minute time constraints. In addition, functional near-infrared spectroscopy (fNIRS) recordings were carried out for each individual from the two groups while playing a 1-minute or 5-minute chess game. It was discovered that under temporal constraints, players exhibited different patterns of functional connectivity in frontal-parietal regions, suggesting that temporal stress can enhance segmentation processes in chess games. In particular, the experienced group exhibited significantly enhanced functional connectivity networks under time pressure including the dorsolateral prefrontal cortex, inferior frontal gyrus, supramarginal gyrus, and postcentral gyrus, which demonstrated the important role of the segmentation process for experienced players under time pressure. Our study found that experienced players were able to enhance recall, reorganize, and integrate chunks to improve chess performance under time pressure.

Proactive interference of visual working memory chunks implicates long-term memory.

Visual working memory (VWM) is a limited cognitive resource that can be functionally expanded through chunking (Miller, 1956). For example, participants can hold an increasing number of colours in mind as they learn to chunk reliably paired combinations (Bradyet al.,2009). We investigated whether this benefit is mediated through the in situ compression of VWM representations (Brady et al.,2009) or the offloading of chunks to long-term memory (LTM; Huang & Awh, 2018; Ngiam et al., 2019) by asking if a vulnerability of LTM - proactive interference - influences VWM performance. We adapted previous designs using deterministic (Experiment 1, N = 60) and probabilistic pairings (Experiments 2 and 3, N = 64 and 80, respectively), to include colour pairings that swapped in sequence along with pairings that were consistent in sequence. Generally, participants reported colours from consistent pairs more accurately than from swapping pairs, which we designed to drive interference in LTM (Experiments 1 and 2). The error profiles also pointed to proactive interference between swapping pairs in all three experiments. Moreover, participants who had explicit awareness of frequent colour pairings had higher VWM accuracy, and their errors reflected more proactive interference than their unaware counterparts (Experiment 3). This pattern of long-term proactive interference in a VWM task lends support for accounts of VWM chunking that propose LTM offloading.

Modeling Effects of Rumination on Free Recall Using ACT-R.

Ruminative thinking, characterized by a recurrent focus on negative and self-related thought, is a key cognitive vulnerability marker of depression and, therefore, a key individual difference variable. This study aimed to develop a computational cognitive model of rumination focusing on the organization and retrieval of information in memory, and how these mechanisms differ in individuals prone to rumination and individuals less prone to rumination. Adaptive Control of Thought-Rational (ACT-R) was used to develop a rumination model by adding memory chunks with negative valence to the declarative memory. In addition, their strength of association was increased to simulate recurrent negative focus, thereby making it harder to disengage from. The ACT-R models were validated by comparing them against two empirical datasets containing data from control and depressed participants. Our general and ruminative models were able to recreate the benchmarks of free recall while matching the behavior exhibited by the control and the depressed participants, respectively. Our study shows that it is possible to build a computational theory of rumination that can accurately simulate the differences in free recall dynamics between control and depressed individuals. Such a model could enable a more fine-tuned investigation of underlying cognitive mechanisms of depression and potentially help to improve interventions by allowing them to more specifically target key mechanisms that instigate and maintain depression.

Using a Computational Cognitive Model to Understand Phishing Classification Decisions of Email Users

Abstract Numerous studies of human user behaviours in cybersecurity tasks have used traditional research methods, such as self-reported surveys or empirical experiments, to identify relationships between various factors of interest and user security performance. This work takes a different approach, applying computational cognitive modelling to research the decision-making of cybersecurity users. The model described here relies on cognitive memory chunk activation to analytically simulate the decision-making process of a user classifying legitimate and phishing emails. Suspicious-seeming cues in each email are processed by examining similar, past classifications in long-term memory. We manipulate five parameters (Suspicion Threshold, Maximum Cues Processed, Weight of Similarity, Flawed Perception Level, Legitimate-to-Phishing Email Ratio in long-term memory) to examine their effects on accuracy, email processing time and decision confidence. Furthermore, we have conducted an empirical, unattended study of US participants performing the same task. Analyses on the empirical study data and simulation output, especially clustering analysis, show that these two research approaches complement each other for more insightful understanding of this phishing detection task. The analyses also demonstrate several limitations of this computational model that cannot easily capture certain user types and phishing detection strategies, calling for a more dynamic and sophisticated model construction.

A Research to Improve Contiguous Memory Allocation in Linux Kernel

The demand for Contiguous Memory Allocation (CMA) has witnessed significant growth in both low-end and high-end devices in recent years. However, the existing practices for utilizing CMA prove insufficient, particularly when catering to the needs of low-end (32-bit) devices. CMA, a Linux program used for memory reservation and allocation, faces limitations in its current implementations. Presently, techniques such as Scatter-Gather Direct Memory Access (DMA), Input Output Memory Management Unit (IOMMU), and Memory Reservation are commonly employed for contiguous memory allocation. Unfortunately, these methods are financially impractical for low-end devices and struggle to efficiently allocate substantial memory chunks, leading to latency concerns. In this paper, we introduce an improved CMA approach that intelligently allocates virtual memory for data mapping as needed. Alternatively, it directly allocates and deallocates physical memory without the necessity of virtual memory mapping, employing the DMA_KERNEL_NO_MAPPING attribute within the DMA Application Programming Interface (API). By adopting this method, latency is reduced, and the facilitation of larger memory allocations is promoted, addressing the limitations of the current techniques.

Parallel Training of Pre-Trained Models via Chunk-Based Dynamic Memory Management

The pre-trained model (PTM) is revolutionizing Artificial Intelligence (AI) technology. However, the hardware requirement of PTM training is prohibitively high, making it a game for a small proportion of people. Therefore, we proposed PatrickStar system to lower the hardware requirements of PTMs and make them accessible to everyone. PatrickStar uses the CPU-GPU heterogeneous memory space to store the model data. Different from existing works, we organize the model data in memory chunks and dynamically distribute them in the heterogeneous memory. Guided by the runtime memory statistics collected in a warm-up iteration, chunks are orchestrated efficiently in heterogeneous memory and generate lower CPU-GPU data transmission volume and higher bandwidth utilization. Symbiosis with the Zero Redundancy Optimizer, PatrickStar scales to multiple GPUs on multiple nodes. % using data parallelism. The system can train tasks on bigger models and larger batch sizes, which cannot be accomplished by existing works. Experimental results show that PatrickStar extends model scales 2.27 and 2.5 times of DeepSpeed, and consistently exhibits significantly higher execution speed. PatricStar also successfully runs the 175B GPT3 training task on a 32 GPU cluster. Our code is publicly available at https://github.com/Tencent/PatrickStar.

Syntactic chunking reveals a core syntactic representation of multi-digit numbers, which is generative and automatic

Representing the base-10 structure of numbers is a challenging cognitive ability, unique to humans, but it is yet unknown how precisely this is done. Here, we examined whether and how literate adults represent a number’s full syntactic structure. In 5 experiments, participants repeated number-word sequences and we systematically varied the order of words within each sequence. Repetition on grammatical sequences (e.g., two hundred ninety-seven) was better than on non-grammatical ones (hundred seven two ninety). We conclude that the participants represented the number’s full syntactic structure and used it to merge number words into chunks in short-term memory. Accuracy monotonously improved for sequences with increasingly longer grammatical segments, up to a limit of ~ 4 words per segment, irrespectively of the number of digits, and worsened thereafter. Namely, short chunks improved memorization, whereas oversized chunks disrupted memorization. This chunk size limit suggests that the chunks are not based on predefined structures, whose size limit is not expected to be so low, but are created ad hoc by a generative process, such as the hierarchical syntactic representation hypothesized in Michael McCloskey’s number-processing model. Chunking occurred even when it disrupted performance, as in the oversized chunks, and even when external cues for chunking were controlled for or were removed. We conclude that the above generative process operates automatically rather than voluntarily. To date, this is the most detailed account of the core representation of the syntactic structure of numbers—a critical aspect of numerical literacy and of the ability to read and write numbers.

A Structure-Aware Storage Optimization for Out-of-Core Concurrent Graph Processing

With the huge demand for graph analytics in many real-world applications, massive iterative graph processing jobs are concurrently performed on the same graphs and suffer from significant high data access cost. To lower the data access cost toward high performance, several out-of-core concurrent graph processing solutions are recently designed to handle concurrent jobs by enabling these jobs to share the accesses of the same graph data. However, the set of active vertices in each partition are usually different for various concurrent jobs and also evolve with time, where some high-degree ones (or called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hub-vertices</i> ) of these active vertices require more iterations to converge due to the power-law property of real-world graphs. In consequence, existing solutions still suffer from much unnecessary I/O traffic, because they have to entirely load each partition into the memory for concurrent jobs even if most vertices in this partition are inactive and may be shared by a few jobs. In this paper, we propose an efficient structure-aware storage system, called GraphSO, for higher throughput of the execution of concurrent graph processing jobs. It can be integrated into existing out-of-core graph processing systems to promote the execution efficiency of concurrent jobs with lower I/O overhead. The key design of GraphSO is a fine-grained storage management scheme. Specifically, it logically divides the partitions of existing graph processing systems into a series of small same-sized chunks. At runtime, these small chunks with active vertices are judiciously loaded by GraphSO to construct new logical partitions (i.e., each logical partition is a subset of active chunks) for existing graph processing systems to handle, where the most-frequently-used chunks are preferentially loaded to construct the logical partitions and the other ones are delayed to wait to be required by more jobs. In this way, it can effectively spare the cost of loading the graph data associated with the inactive vertices with low repartitioning overhead and can also enable the loaded graph data to be fully shared by concurrent jobs. Moreover, GraphSO also designs a buffering strategy to efficiently cache the most-frequently-used chunks in the main memory to further minimize the I/O traffic by avoiding repeated load of them. Experimental results show that GraphSO improves the throughput of GridGraph, GraphChi, X-Stream, DynamicShards, LUMOS, Graphene, and Wonderland by 1.4-3.5 times, 2.1-4.3 times, 1.9-4.1 times, 1.9-2.9 times, 1.5-3.1 times, 1.3-1.5 times, and 1.3-2.7 times after integrating with them, respectively.

Dynamic Memory Management in Massively Parallel Systems: A Case on GPUs.

Due to the high level of parallelism, there are unique challenges in developing system software on massively parallel hardware such as GPUs. One such challenge is designing a dynamic memory allocator whose task is to allocate memory chunks to requesting threads at runtime. State-of-the-art GPU memory allocators maintain a global data structure holding metadata to facilitate allocation/deallocation. However, the centralized data structure can easily become a bottleneck in a massively parallel system. In this paper, we present a novel approach for designing dynamic memory allocation without a centralized data structure. The core idea is to let threads follow a random search procedure to locate free pages. Then we further extend to more advanced designs and algorithms that can achieve an order of magnitude improvement over the basic idea. We present mathematical proofs to demonstrate that (1) the basic random search design achieves asymptotically lower latency than the traditional queue-based design and (2) the advanced designs achieve significant improvement over the basic idea. Extensive experiments show consistency to our mathematical models and demonstrate that our solutions can achieve up to two orders of magnitude improvement in latency over the best-known existing solutions.

B2-Tree: Page-Based String Indexing in Concurrent Environments

Recently proposed index structures, that combine trie-based and comparison-based search mechanisms, considerably improve retrieval throughput for in-memory database systems. However, most of these index structures allocate small memory chunks when required. This stands in contrast to block-based index structures, that are necessary for disk-accesses of beyond main-memory database systems such as Umbra. We therefore present the B2-tree. The outer structure is identical to that of an ordinary B+-tree. It still stores elements in a dense array in sorted order, enabling efficient range scan operations. However, B2-tree is composed of multiple trees, each page integrates another trie-based search tree, which is used to determine a small memory region where a sought entry may be found. An embedded tree thereby consists of decision nodes, which operate on a single byte at a time, and span nodes, which are used to store common prefixes. This architecture usually accesses fewer cache lines than a vanilla B+-tree as shown in our performance evaluation. As a result, the B2-tree answers point queries considerably faster. Concurrent access to B2-tree pages are managed by an optimistic locking protocol which results in high utilization of the available hardware resources. Our evaluation of read-write workloads attests more than competitive performance for the B2-tree compared to a traditional B+-tree.

Working Memory for Online Memory Binding Tasks: A Hybrid Model

Working memory is the brain module that holds and manipulates information online. In this work, we design a hybrid model in which a simple feed-forward network is coupled to a balanced random network via a read–write vector called the interface vector. Three cases and their results are discussed similar to the n-back task called, first-order memory binding task, generalized first-order memory task, and second-order memory binding task. The important result is that our dual-component model of working memory shows good performance with learning restricted to the feed-forward component only. Here, we take advantage of the random network property without learning. Finally, a more complex memory binding task called, a cue-based memory binding task, is introduced in which a cue is given as input representing a binding relation that prompts the network to choose the useful chunk of memory. To our knowledge, this is the first time that random networks as a flexible memory is shown to play an important role in online binding tasks. We may interpret our results as a candidate model of working memory in which the feed-forward network learns to interact with the temporary storage random network as an attentional-controlling executive system.

Eye Movements of Spatial Working Memory Encoding in Children with and without Autism: Chunking Processing and Reference Preference.

Individuals with autism spectrum disorder (ASD) experience spatial working memory deficits and show different encoding mechanisms from typical developing (TD) peers. To effectively describe the encoding strategies of those with ASD and highlight their characteristics in cognitive processing, we adopted improved change detection tasks and added eye-movement indicators to investigate the chunking function and reference preference of children with and without ASD. The current study included 20 participants with ASD aged 8-16 and 20 TD children matched for age, gender, and intelligence. Experiment 1 used high/low-structured change detection tasks, and eye-movement indexes were recorded as they memorized the locations of the items to investigate spatial chunking strategies. In Experiment 2, changes in eye movement patterns were observed by adding a frame of reference. The results suggested different encoding strategies in ASD and TD individuals. The ASD group showed local processing bias and had difficulty adopting chunking strategies in spatial working memory. Eye-movement analysis suggested that they rarely showed integrated information processing tendency observed in TD children. Moreover, as a compensatory processing, they were more likely to use the frame of reference. In this study, we compared the spatial chunking strategies and reference preference of children with and without ASD, and eye-movement analysis was used to investigate the processing mechanism. These findings are significant for research on cognitive characteristics of ASD and provide a new focus for working memory training in children with ASD. LAY SUMMARY: The current study suggests that children with autism spectrum disorder are poorer at organizing items into chunks in spatial working memory, but rely more on reference frames. If the purpose of location memory is to strengthen the adaptability of children with autism, it should provide them with more clues or references. If it is for the purpose of intervention such as cognitive training, it should guide them to integrate information to improve the basic cognitive processing efficiency. Autism Res 2021, 14: 897-910. © 2020 International Society for Autism Research and Wiley Periodicals LLC.

Statistical learning and memory

Learners often need to identify and remember recurring units in continuous sequences, but the underlying mechanisms are debated. A particularly prominent candidate mechanism relies on distributional statistics such as Transitional Probabilities (TPs). However, it is unclear what the outputs of statistical segmentation mechanisms are, and if learners store these outputs as discrete chunks in memory. We critically review the evidence for the possibility that statistically coherent items are stored in memory and outline difficulties in interpreting past research. We use Slone and Johnson's (2018) experiments as a case study to show that it is difficult to delineate the different mechanisms learners might use to solve a learning problem. Slone and Johnson (2018) reported that 8-month-old infants learned coherent chunks of shapes in visual sequences. Here, we describe an alternate interpretation of their findings based on a multiple-cue integration perspective. First, when multiple cues to statistical structure were available, infants' looking behavior seemed to track with the strength of the strongest one — backward TPs, suggesting that infants process multiple cues simultaneously and select the strongest one. Second, like adults, infants are exquisitely sensitive to chunks, but may require multiple cues to extract them. In Slone and Johnson's (2018) experiments, these cues were provided by immediate chunk repetitions during familiarization. Accordingly, infants showed strongest evidence of chunking following familiarization sequences in which immediate repetitions were more frequent. These interpretations provide a strong argument for infants' processing of multiple cues and the potential importance of multiple cues for chunk recognition in infancy.

Electroencephalographic response of chess players in decision-making processes under time pressure

The game of chess offers an enabling environment for the study of basic cognitive decision making processes where different types of memories are involved. The aim of the present study was to describe and compare the brain electrical activity pattern of chess players in two different situations under time pressure to make the move: rapid, 15 min, plus 10 s additional time per move and lightning, a one-minute game. Thus, lightning game offers an appropriate environment to isolate memory chunks and rapid game will offer the perspective of a working memory task. Encephalogram (EEG) recordings were obtained while a total of 14 male chess players (age: 35.36 ± 13.77 and ELO: 1921.07 ± 170.67) played lightning and rapid chess games. Results indicate that different activity patterns between the rapid and the lightning games were found. Increments in theta power during lightning game in posterior regions could indicate that theta power might be related with chunks retrieval. Moreover, right hemisphere is more activated in chess games, probably caused by visuospatial processing. These brain patterns characterizations could be useful for chess training or clinicians in the field of neuroscience.

How Does Rumination Impact Cognition? A First Mechanistic Model.

Rumination is a process of uncontrolled, narrowly focused negative thinking that is often self-referential, and that is a hallmark of depression. Despite its importance, little is known about its cognitive mechanisms. Rumination can be thought of as a specific, constrained form of mind-wandering. Here, we introduce a cognitive model of rumination that we developed on the basis of our existing model of mind-wandering. The rumination model implements the hypothesis that rumination is caused by maladaptive habits of thought. These habits of thought are modeled by adjusting the number of memory chunks and their associative structure, which changes the sequence of memories that are retrieved during mind-wandering, such that during rumination the same set of negative memories is retrieved repeatedly. The implementation of habits of thought was guided by empirical data from an experience sampling study in healthy and depressed participants. On the basis of this empirically derived memory structure, our model naturally predicts the declines in cognitive task performance that are typically observed in depressed patients. This study demonstrates how we can use cognitive models to better understand the cognitive mechanisms underlying rumination and depression.

Pushing typists back on the learning curve: Memory chunking improves retrieval of prior typing episodes.

Hierarchical control of skilled performance depends on chunking of several lower-level units into a single higher-level unit. The present study examined the relationship between chunking and recognition of trained materials in the context of typewriting. In 3 experiments, participants were trained with typing nonwords and were later tested on their recognition of the trained materials. In Experiment 1, participants typed the same words or nonwords in 5 consecutive trials while performing a concurrent memory task. In Experiment 2, participants typed the materials with lags between repetitions without a concurrent memory task. In both experiments, recognition of typing materials was associated with better chunking of the materials. Experiment 3 used the remember-know procedure to test the recollection and familiarity components of recognition. Remember judgments were associated with better chunking than know judgments or nonrecognition. These results indicate that chunking is associated with explicit recollection of prior typing episodes. The relevance of the existing memory models to chunking in typewriting was considered, and it is proposed that memory chunking improves retrieval of trained typing materials by integrating contextual cues into the memory traces. (PsycINFO Database Record

Hybrid TLB Coalescing

To mitigate excessive TLB misses in large memory applications, techniques such as large pages, variable length segments, and HW coalescing, increase the coverage of limited hardware translation entries by exploiting the contiguous memory allocation. However, recent studies show that in non-uniform memory systems, using large pages often leads to performance degradation, or allocating large chunks of memory becomes more difficult due to memory fragmentation. Although each of the prior techniques favors its own best chunk size, diverse contiguity of memory allocation in real systems cannot always provide the optimal chunk of each technique. Under such fragmented and diverse memory allocations, this paper proposes a novel HW-SW hybrid translation architecture, which can adapt to different memory mappings efficiently. In the proposed hybrid coalescing technique, the operating system encodes memory contiguity information in a subset of page table entries, called anchor entries. During address translation through TLBs, an anchor entry provides translation for contiguous pages following the anchor entry. As a smaller number of anchor entries can cover a large portion of virtual address space, the efficiency of TLB can be significantly improved. The most important benefit of hybrid coalescing is its ability to change the coverage of the anchor entry dynamically, reflecting the current allocation contiguity status. By using the contiguity information directly set by the operating system, the technique can provide scalable translation coverage improvements with minor hardware changes, while allowing the flexibility of memory allocation. Our experimental results show that across diverse allocation scenarios with different distributions of contiguous memory chunks, the proposed scheme can effectively reap the potential translation coverage improvement from the existing contiguity.

Oscillatory EEG dynamics underlying automatic chunking during sentence processing

Sentences are easier to remember than random word sequences, likely because linguistic regularities facilitate chunking of words into meaningful groups. The present electroencephalography study investigated the neural oscillations modulated by this so-called sentence superiority effect during the encoding and maintenance of sentence fragments versus word lists. We hypothesized a chunking-related modulation of neural processing during the encoding and retention of sentences (i.e., sentence fragments) as compared to word lists. Time–frequency analysis revealed a two-fold oscillatory pattern for the memorization of sentences: Sentence encoding was accompanied by higher delta amplitude (4Hz), originating both from regions processing syntax as well as semantics (bilateral superior/middle temporal regions and fusiform gyrus). Subsequent sentence retention was reflected in decreased theta (6Hz) and beta/gamma (27–32Hz) amplitude instead. Notably, whether participants simply read or properly memorized the sentences did not impact chunking-related activity during encoding. Therefore, we argue that the sentence superiority effect is grounded in highly automatized language processing mechanisms, which generate meaningful memory chunks irrespective of task demands.

Enhancing the Malloc System with Pollution Awareness for Better Cache Performance

Cache pollution, by which weak-locality data unduly replaces strong-locality data, may notably degrade application performance in a shared-cache multicore machine. This paper presents NightWatch, a cache management subsystem that provides general, transparent and low-overhead pollution control to applications. NightWatch is based on the observation that data within the same memory chunk or chunks within the same allocation context often share similar locality property. NightWatch embodies this observation by online monitoring current cache locality to predict future behavior and restricting potential cache polluters proactively. We have integrated NightWatch into two popular allocators, tcmalloc and ptmalloc2 . Experiments with SPEC CPU2006 show that NightWatch improves application performance by up to 45 percent (18 percent on average), with an average monitoring overhead of 0.57 percent (up to 3.02 percent).

On the Restore Time Variations of Future DRAM Memory

As the de facto main memory standard, DRAM (Dynamic Random Access Memory) has achieved dramatic density improvement in the past four decades, along with the advancements in process technology. Recent studies reveal that one of the major challenges in scaling DRAM into the deep sub-micron regime is its significant variations on cell restore time, which affect timing constraints such as write recovery time. Adopting traditional approaches results in either low yield rate or large performance degradation. In this article, we propose schemes to expose the variations to the architectural level. By constructing memory chunks with different access speeds and, in particular, exploiting the performance benefits of fast chunks, a variation-aware memory controller can effectively mitigate the performance loss due to relaxed timing constraints. We then proposed restore-time-aware rank construction and page allocation schemes to make better use of fast chunks. Our experimental results show that, compared to traditional designs such as row sparing and Error Correcting Codes, the proposed schemes help to improve system performance by about 16% and 20%, respectively, for 20nm and 14nm technology nodes on a four-core multiprocessor system.