Allocation Of Memory Resources Research Articles

Adaptive forgetting speed in working memory.

Working memory is known to be capacity-limited and is therefore selective not only for what it encodes but also what it forgets. Explicit forgetting cues can be used effectively to free up capacity, but it is not clear how working memory adaptively forgets in the absence of explicit cues. An important implicit cue that may tune forgetting in working memory is the passage of time. When information becomes irrelevant more quickly, working memory should also forget information more quickly. In three delayed-estimation experiments, we systematically manipulated how probing probability changed as time passed on after encoding an item (i.e., the "probing hazard"). In some blocks, probing hazard decreased after encoding an item, requiring participants to only briefly retain the memory item. In other blocks, the probing hazard increased or stayed flat, as the retention interval was lengthened. In line with our hypothesis, we found that participants adapted their forgetting rate to the probing dynamics of the working memory task. When the memory item quickly became irrelevant ("decreasing" probing hazard), forgetting rate was higher than in blocks where probing hazard increased or stayed flat. The time course of these adaptations in forgetting implies a fast and flexible mechanism. Interestingly, participants could not explicitly report the order of conditions, suggesting forgetting is implicitly sped up. These findings suggest that implicit adaptations to the temporal structure of our environment tune forgetting speed in working memory, possibly contributing to the flexible allocation of limited working memory resources.

Heterogeneous Forgetting Rates and Greedy Allocation in Slot-Based Memory Networks Promotes Signal Retention.

A key question in the neuroscience of memory encoding pertains to the mechanisms by which afferent stimuli are allocated within memory networks. This issue is especially pronounced in the domain of working memory, where capacity is finite. Presumably the brain must embed some "policy" by which to allocate these mnemonic resources in an online manner in order to maximally represent and store afferent information for as long as possible and without interference from subsequent stimuli. Here, we engage this question through a top-down theoretical modeling framework. We formally optimize a gating mechanism that projects afferent stimuli onto a finite number of memory slots within a recurrent network architecture. In the absence of external input, the activity in each slot attenuates over time (i.e., a process of gradual forgetting). It turns out that the optimal gating policy consists of a direct projection from sensory activity to memory slots, alongside an activity-dependent lateral inhibition. Interestingly, allocating resources myopically (greedily with respect to the current stimulus) leads to efficient utilization of slots over time. In other words, later-arriving stimuli are distributed across slots in such a way that the network state is minimally shifted and so prior signals are minimally "overwritten." Further, networks with heterogeneity in the timescales of their forgetting rates retain stimuli better than those that are more homogeneous. Our results suggest how online, recurrent networks working on temporally localized objectives without high-level supervision can nonetheless implement efficient allocation of memory resources over time.

Limitations on flexible allocation of visual short-term memory resources with multiple levels of goal-directed attentional prioritization.

Studies suggest that visual short-term memory (VSTM) is a continuous resource that can be flexibly allocated using probabilistic cues that indicate test likelihood (i.e., goal-directed attentional priority to those items). Previous studies using simultaneous cues have not examined this flexible allocation beyond two distinct levels of priority. Moreover, previous studies have not examined whether there are individual differences in the ability to flexibly allocate VSTM resources, as well as whether this ability benefits from practice. The current study used a continuous report procedure to examine whether participants can use up to three levels of attentional priority to allocate VSTM resources via simultaneous probabilistic spatial cues. Three experiments were performed with differing priority levels, cues, and cue presentation times. Group level analysis demonstrated flexible allocation of VSTM resources; however, there was limited evidence that participants could use three goal-directed priority levels. A temporal analysis suggested that task fatigue, rather than practice effects, may interact with item priority. A Bayesian individual-differences analysis revealed that a minority of participants were using three levels of attentional priority, demonstrating that, while possible, it is not the predominant pattern of behaviour. Thus, we provided evidence that flexible allocation to three attention levels is possible under simultaneous cuing conditions for a minority of participants. Flexible allocation to three categories may be interpreted as a skill of high-performing participants akin to high memory capacity.

Optimal FPGA memory allocation for image processing

In the field of computer vision, Field Programmable Gate Array (FGPA) limited de on-chip memory is difficult to meet the power, size and other requirements. To address this phenomenon, the study constructs a partitioning algorithm to achieve a balance between energy consumption and resource utilisation based on the analysis of memory resource allocation, overall power consumption and resource utilisation from the perspective of image processing technology. The power consumption of the balancing algorithm is lower compared to the optimised utilisation algorithm HLS tool, with both Block Ramdom Access Memory (BRAM) power consumption taking the value of 0.005; the dynamic power consumption takes the value range of 0.014–0.082. Compared to the High Level Synthesis (HLS) tool, the overall power consumption of the balancing algorithm and the optimised utilisation algorithm is significantly lower, with the values of 0.251 and 0.252 respectively, both with a reduction rate of approximately 30%. The accuracy rate of the proposed memory optimisation allocation algorithm is the highest among the four memory optimisation allocation algorithms and strategies on all three types of target scales. FPGA memory optimisation allocation strategy can guarantee to have lower power consumption while satisfying the same resource occupancy, and the model has in-depth application value in visual image vision technology.

Neural Network Compression for Noisy Storage Devices

Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices. Despite the significant progress in NN model compression, there has been considerably less investigation in the actual physical storage of NN parameters. Conventionally, model compression and physical storage are decoupled, as digital storage media with error-correcting codes (ECCs) provide robust error-free storage. However, this decoupled approach is inefficient as it ignores the overparameterization present in most NNs and forces the memory device to allocate the same amount of resources to every bit of information regardless of its importance. In this work, we investigate analog memory devices as an alternative to digital media – one that naturally provides a way to add more protection for significant bits unlike its counterpart, but is noisy and may compromise the stored model’s performance if used naively. We develop a variety of robust coding strategies for NN weight storage on analog devices, and propose an approach to jointly optimize model compression and memory resource allocation. We then demonstrate the efficacy of our approach on models trained on MNIST, CIFAR-10, and ImageNet datasets for existing compression techniques. Compared to conventional error-free digital storage, our method reduces the memory footprint by up to one order of magnitude, without significantly compromising the stored model’s accuracy.

A High-Throughput Full-Dataflow MobileNetv2 Accelerator on Edge FPGA

FPGA accelerators for lightweight neural networks such as MobileNetv2 are of great need in edge computing applications with high throughput requirements. Dataflow architecture has been considered a promising approach to optimize throughput since the intermediate feature map transfers can be significantly saved. However, previous MobileNetv2 accelerators only achieved a partial-dataflow architecture, and just one-third of the feature map transfers can be saved. To solve this issue, we propose a scheme to achieve a full-dataflow MobileNetv2 accelerator on FPGA. The scheme contains four techniques. First, we improve the full-integer quantization for easier deployment on hardware. Second, we propose tunable activation weight imbalance transfer for less quantization accuracy loss. Third, we present several highly optimized accelerator components whose parallelism can be flexibly adjusted, and implement residual connection with deeper FIFO so that the requirements of the full-dataflow architecture can be fully met. Finally, we present a computing resource allocation strategy to balance the latency of each layer, and a memory resource allocation strategy to effectively use the on-chip memory. Compared to the state-ofthe-art, experimental results show that the accelerator achieves 1910 FPS with 1.8× speedup when implemented on the Xilinx ZCU102 FPGA. In addition, it reaches 72.98% Top-1 accuracy with 8-bit integer quantization that outperforms all the other MobileNetv2 accelerators.

Increasing perceptual separateness affects working memory for depth - re-allocation of attention from boundaries to the fixated center.

For decades, working memory (WM) has been a heated research topic in the field of cognitive psychology. However, most studies on WM presented visual stimuli on a two-dimensional plane, rarely involving depth perception. Several previous studies have investigated how depth information is stored in WM, and found that WM for depth is even more limited in capacity and the memory performance is poor compared to visual WM. In the present study, we used a change detection task to investigate whether dissociating memory items by different visual features, thereby to increase their perceptual separateness, can improve WM performance for depth. Memory items presented at various depth planes were bound with different colors (Experiments 1 and 3) or sizes (Experiment 2). The memory performance for depth locations of visual stimuli with homogeneous and heterogeneous appearances were tested and compared. The results showed a consistent pattern that although separating items with various feature values did not affect the overall memory performance, the manipulation significantly improved memory performance for the middle depth locations but impaired the performance for the boundary locations when observers fixated at the center of the whole depth volume. The memory benefits of feature separation can be attributed to enhanced individuation of memory items, therefore facilitating a more balanced allocation of attention and memory resources.

QuickLD: An efficient software for linkage disequilibrium analyses.

Software tools for linkage disequilibrium (LD) analyses are designed to calculate LD among all genetic variants in a single region. Since compute and memory requirements grow quadratically with the distance between variants, using these tools for long‐range LD calculations leads to long execution times and increased allocation of memory resources. Furthermore, widely used tools do not fully utilize the computational resources of modern processors and/or graphics processing cards, limiting future large‐scale analyses on thousands of samples. We present quickLD, a stand‐alone and open‐source software that computes several LD‐related statistics, including the commonly used r 2. quickLD calculates pairwise LD between genetic variants in a single region or in arbitrarily distant regions with negligible memory requirements. Moreover, quickLD achieves up to 95% and 97% of the theoretical peak performance of a CPU and a GPU, respectively, enabling 21.5× faster processing than current state‐of‐the‐art software on a multicore processor and 49.5× faster processing when the aggregate processing power of a multicore CPU and a GPU is harnessed. quickLD can also be used in studies of selection, recombination, genetic drift, inbreeding and gene flow. The software is available at https://github.com/pephco/quickLD.

The Effects of Task Difficulty Predictability and Noise Reduction on Recall Performance and Pupil Dilation Responses.

Objectives:Communication requires cognitive processes which are not captured by traditional speech understanding tests. Under challenging listening situations, more working memory resources are needed to process speech, leaving fewer resources available for storage. The aim of the current study was to investigate the effect of task difficulty predictability, that is, knowing versus not knowing task difficulty in advance, and the effect of noise reduction on working memory resource allocation to processing and storage of speech heard in background noise. For this purpose, an “offline” behavioral measure, the Sentence-Final Word Identification and Recall (SWIR) test, and an “online” physiological measure, pupillometry, were combined. Moreover, the outcomes of the two measures were compared to investigate whether they reflect the same processes related to resource allocation.Design:Twenty-four experienced hearing aid users with moderate to moderately severe hearing loss participated in this study. The SWIR test and pupillometry were measured simultaneously with noise reduction in the test hearing aids activated and deactivated in a background noise composed of four-talker babble. The task of the SWIR test is to listen to lists of sentences, repeat the last word immediately after each sentence and recall the repeated words when the list is finished. The sentence baseline dilation, which is defined as the mean pupil dilation before each sentence, and task-evoked peak pupil dilation (PPD) were analyzed over the course of the lists. The task difficulty predictability was manipulated by including lists of three, five, and seven sentences. The test was conducted over two sessions, one during which the participants were informed about list length before each list (predictable task difficulty) and one during which they were not (unpredictable task difficulty).Results:The sentence baseline dilation was higher when task difficulty was unpredictable compared to predictable, except at the start of the list, where there was no difference. The PPD tended to be higher at the beginning of the list, this pattern being more prominent when task difficulty was unpredictable. Recall performance was better and sentence baseline dilation was higher when noise reduction was on, especially toward the end of longer lists. There was no effect of noise reduction on PPD.Conclusions:Task difficulty predictability did not have an effect on resource allocation, since recall performance was similar independently of whether task difficulty was predictable or unpredictable. The higher sentence baseline dilation when task difficulty was unpredictable likely reflected a difference in the recall strategy or higher degree of task engagement/alertness or arousal. Hence, pupillometry captured processes which the SWIR test does not capture. Noise reduction frees up resources to be used for storage of speech, which was reflected in the better recall performance and larger sentence baseline dilation toward the end of the list when noise reduction was on. Thus, both measures captured different temporal aspects of the same processes related to resource allocation with noise reduction on and off.

Cooperative Game-Based Virtual Machine Resource Allocation Algorithms in Cloud Data Centers

With the growing demand of cloud services, cloud data centers (CDCs) can provide flexible resource provisioning in order to accommodate the workload demand. In CDCs, the virtual machine (VM) resource allocation problem is an important and challenging issue to provide efficient infrastructure services. In this paper, we propose a unified resource allocation scheme for VMs in the CDC system. To provide a fair-efficient solution, we concentrate on the basic concept of Shapley value and adopt its variations to effectively allocate CDC resources. Based on the characteristics of value solutions, we develop novel CPU, memory, storage, and bandwidth resource allocation algorithms. To practically implement our algorithms, application types are assumed as cooperative game players, and different value solutions are applied to optimize the resource utilization. Therefore, our four resource allocation algorithms are jointly combined as a novel fourfold game model and take various benefits in a rational way through the cascade interactions while solving comprehensively some control issues. To ensure the growing demand of cloud services, this feature can leverage the full synergy of different value solutions. To check the effectiveness and superiority of our proposed scheme, we conduct extensive simulations. The simulation results show that our algorithms have significant performance improvement compared to the existing state-of-the-art protocols. Finally, we summarize our cooperative game-based approach and discuss possible major research issues for the future challenges about the cloud-assisted DC resource allocation paradigm.

Electrophysiological correlates of the flexible allocation of visual working memory resources

Visual working memory is a brief, capacity-limited store of visual information that is involved in a large number of cognitive functions. To guide one’s behavior effectively, one must efficiently allocate these limited memory resources across memory items. Previous research has suggested that items are either stored in memory or completely blocked from memory access. However, recent behavioral work proposes that memory resources can be flexibly split across items based on their level of task importance. Here, we investigated the electrophysiological correlates of flexible resource allocation by manipulating the distribution of resources amongst systematically lateralized memory items. We examined the contralateral delay activity (CDA), a waveform typically associated with the number of items held in memory. Across three experiments, we found that, in addition to memory load, the CDA flexibly tracks memory resource allocation. This allocation occurred as early as attentional selection, as indicated by the N2pc. Additionally, CDA amplitude was better-described when fit with a continuous model predicted by load and resources together than when fit with either alone. Our findings show that electrophysiological markers of attentional selection and memory maintenance not only track memory load, but also the proportion of memory resources those items receive.

Improving resource allocation in software-defined networks using clustering

Software-defined networks (SDNs) are a huge evolution in simplifying implementation and network operation which have reduced costs and made the network programmable. Although SDNs are a suitable option for solving some of the previous problems, but they have some challenges as any other new technology. Resource allocation and balance control in network is one of the main challenges of this technology which is studied in this paper. In this study, a new approach is proposed for improving memory resource allocation in network using load distribution clusters. Since in the proposed method, K-mean++ algorithm is used for clustering, load balancing of clusters can be used to preserve load balance of the network. In the proposed method, data with higher recall is transmitted to high-quality clusters in terms of average number of hubs and lower average delay between server and user. In the proposed method, by increasing number of clusters, higher memory is created in the network.

A Dynamic Memory Allocation Approach Based on Balloon Technology on Virtualization Platform

In virtualization platforms, Virtual Machines(VMs) have different memory demand when running different applications. Applications with high memory demand often result in guest swapping and performance degradation of the VM. Because of the semantic gap between the Virtual Machine Monitor(VMM) and the guest Operating System(OS), it is impossible to achieve reasonable allocation and use of memory resources between the guests. Our goal is to monitor the memory information in real time and achieve memory recovery and replenish of each VM in case of memory fluctuation. This method builds up a reasonable swapping mechanism, which can achieve automatic balance of memory allocation among multiple VMs, optimize guest swapping and improve memory utilization. We display it by visualization module. Based on ballooning technology, this paper designs a dynamic memory allocation method in KVM virtualization platform, which consists of memory monitoring, memory dynamic allocation, memory resource visualization and other modules. The results of many tests and analysis demonstrate that our method has the ability to basically achieve our expected goals and requirements.

Optimized Memory Allocation and Power Minimization for FPGA-Based Image Processing.

Memory is the biggest limiting factor to the widespread use of FPGAs for high-level image processing, which require complete frame(s) to be stored in situ. Since FPGAs have limited on-chip memory capabilities, efficient use of such resources is essential to meet performance, size and power constraints. In this paper, we investigate allocation of on-chip memory resources in order to minimize resource usage and power consumption, contributing to the realization of power-efficient high-level image processing fully contained on FPGAs. We propose methods for generating memory architectures, from both Hardware Description Languages and High Level Synthesis designs, which minimize memory usage and power consumption. Based on a formalization of on-chip memory configuration options and a power model, we demonstrate how our partitioning algorithms can outperform traditional strategies. Compared to commercial FPGA synthesis and High Level Synthesis tools, our results show that the proposed algorithms can result in up to 60% higher utilization efficiency, increasing the sizes and/or number of frames that can be accommodated, and reduce frame buffers’ dynamic power consumption by up to approximately 70%. In our experiments using Optical Flow and MeanShift Tracking, representative high-level algorithms, data show that partitioning algorithms can reduce total power by up to 25% and 30%, respectively, without impacting performance.

Optimally Removing Synchronization Overhead for CNNs in Three-Dimensional Neuromorphic Architecture

Recent three-dimensional (3-D) neuromorphic processing-in-memory (PIM) architecture provides a promising hardware-based solution to speed up the processing of convolutional neural networks. However, the limited capacity of the global buffer in this architecture is unable to efficiently handle synchronization overhead. In this paper, we jointly optimize the allocation of computation and memory resources on the 3-D-stacked PIM architecture. The objective is to minimize schedule length by removing synchronization overhead. To guarantee the generation of a feasible task schedule, we theoretically obtain the upper bound to reschedule each computation task. The target problem is further formulated as a dynamic programming model to get an optimal solution. We evaluate our technique with a variety of realistic neural network applications running on deep learning frameworks Caffe and TensorFlow. The results show that the proposed technique can achieve a significant reduction in processing time and improve the utilization of processing cores compared to previous studies.

Hi-DMM: High-Performance Dynamic Memory Management in High-Level Synthesis

High-level synthesis (HLS) of field programmable gate array (FPGA)-based accelerators has been proposed in order to simplify accelerator design process with respect to design time and complexity. However, modern HLS tools do not consider dynamic memory allocation constructs in high-level programming languages like C and limit themselves to static memory allocation. This paper proposes a dynamic memory allocation and management scheme, called Hi-DMM, for inclusion in commercial HLS design flows. Hi-DMM performs source-to-source transformation of user C code with dynamic memory constructs into C-source code with the dynamic memory allocator and management scheme developed in this paper. The transformed C-source code is amenable to synthesis by commercial tools like Vivado HLS. Relying on buddy tree-based allocation schemes and efficient hardware implementation of the allocators, Hi-DMM achieves 4x speed-up in both fine-grained and coarse-grained memory allocation compared to previous works. Experimental results obtained by including Hi-DMM with Vivado-HLS show that dynamic memory allocation of FPGA memory resources can be achieved at a much lower latency with minimal resource overhead, paving the way for synthesis of dynamic memory constructs in commercial HLS flows.

A time division multiplexing algorithm based on FPGA buffer shift to realize high speed image processing

With the development of medical image technology, two-dimensional imaging technology bottlenecks highlighted, three-dimensional imaging technology needs more and more urgent. Cone beam projection computer reorganization tomography equipment principle, which is also called “CBCT”, is the X-ray generator with a lower ray around the projection body to do a circular DR digital projection. Then the data obtained in the “intersection” after projection of the projection body has reconstructed in the computer to obtain the three-dimensional image. Image processing has still faced with image processing problems, such as processing speed and stability. In this paper, the key technology of high-speed image processing system based on FPGA is studied, and a high-speed and high-definition cone-shaped beam projection computerized tomo-graphic image processing platform based on FPGA is realized. The design of this paper is based on the image processing algorithm module of the pipeline. According to the structural characteristics of FPGA, a time division-multiplexing algorithm based on buffer shift is proposed to solve the problem of time division multiplexing for shared resources in image processing process, which facilitates the allocation of buffer memory resources. In this paper, the parallel CORDIC algorithm has introduced to solve the problem of the transcendental function in the image algorithm. The iterative speed advantage is obvious, which provides a powerful guarantee for the later image registration.

Mindfulness-based training with transcranial direct current stimulation modulates neuronal resource allocation in working memory: A randomized pilot study with a nonequivalent control group

Mindfulness-based training (MBT) and transcranial electrical stimulation (TES) methods such as direct current stimulation (tDCS) have demonstrated promise for the augmentation of cognitive abilities. The current study investigated the potential compatibility of concurrent “electrical” MBT and tDCS (or eMBT) by testing its combined effects on behavioral and neurophysiological indices of working memory (WM) and attentional resource allocation. Thirty-four healthy participants were randomly assigned to either a MBT task with tDCS group (eMBT) or an active control training task with sham tDCS (Control) group. Training lasted 4-weeks, with up to twenty MBT sessions and with up to eight of those sessions that were eMBT sessions. Electroencephalography was acquired during varying WM load conditions using the n-back task (1-, 2-, 3-back), along with performance on complex WM span tasks (operation and symmetry span) and fluid intelligence measures (Ravens and Shipley) before and after training. Improved performance was observed only on the 3-back and spatial span tasks for eMBT but not the Control group. During 3-back performance in the eMBT group, an increase in P3 amplitude and theta power at electrode site Pz was also observed, along with a simultaneous decrease in frontal midline P3 amplitude and theta power compared to the Control group. These results are consistent with the neural efficiency hypothesis, where higher cognitive capacity was associated with more distributed brain activity (i.e., increase in parietal and decrease in frontal amplitudes). Future longitudinal studies are called upon to further examine the direct contributions of tDCS on MBT by assessing the differential effects of electrode montage, polarity, current strength and a direct contrast between the eMBT and MBT conditions on performance and neuroimaging outcome data. While preliminary, the current results provided evidence for the potential compatibility of using eMBT to modulate WM capacity through the allocation of attention and its neurophysiological correlates.

Attention mediates the flexible allocation of visual working memory resources.

Though it is clear that it is impossible to store an unlimited amount of information in visual working memory (VWM), the limiting mechanisms remain elusive. While several models of VWM limitations exist, these typically characterize changes in performance as a function of the number of to-be-remembered items. Here, we examine whether changes in spatial attention could better account for VWM performance, independent of load. Across 2 experiments, performance was better predicted by the prioritization of memory items (i.e., attention) than by the number of items to be remembered (i.e., memory load). This relationship followed a power law, and held regardless of whether performance was assessed based on overall precision or any of 3 measures in a mixture model. Moreover, at large set sizes, even minimally attended items could receive a small proportion of resources, without any evidence for a discrete-capacity on the number of items that could be maintained in VWM. Finally, the observed data were best fit by a variable-precision model in which response error was related to the proportion of resources allocated to each item, consistent with a model of VWM in which performance is determined by the continuous allocation of attentional resources during encoding. (PsycINFO Database Record

A Load-Balanced Call Admission Controller for IMS Cloud Computing

Network functions virtualization provides opportunities to design, deploy, and manage networking services. It utilizes cloud computing virtualization services that run on high-volume servers, switches, and storage hardware to virtualize network functions. Virtualization techniques can be used in IP multimedia subsystem (IMS) cloud computing to develop different networking functions (e.g., load balancing and call admission control). IMS network signaling happens through session initiation protocol (SIP). An open issue is the control of overload that occurs when an SIP server lacks sufficient CPU and memory resources to process all messages. This paper proposes a virtual load balanced call admission controller (VLB-CAC) for the cloud-hosted SIP servers. VLB-CAC determines the optimal call admission rates and signaling paths for admitted calls along with the optimal allocation of CPU and memory resources of the SIP servers. This optimal solution is derived through a new linear programming model. This model requires some critical information of SIP servers as input. Further, VLB-CAC is equipped with an autoscaler to overcome resource limitations. The proposed scheme is implemented in smart applications on virtual infrastructure (SAVI) which serves as a virtual testbed. An assessment of the numerical and experimental results demonstrates the efficiency of the proposed work.