Memory Architecture Research Articles

Deep learning for monthly rainfall–runoff modelling: a large-sample comparison with conceptual models across Australia

Abstract. A deep learning model designed for time series predictions, the long short-term memory (LSTM) architecture, is regularly producing reliable results in local and regional rainfall–runoff applications around the world. Recent large-sample hydrology studies in North America and Europe have shown the LSTM model to successfully match conceptual model performance at a daily time step over hundreds of catchments. Here we investigate how these models perform in producing monthly runoff predictions in the relatively dry and variable conditions of the Australian continent. The monthly time step matches historic data availability and is also important for future water resources planning; however, it provides significantly smaller training datasets than daily time series. In this study, a continental-scale comparison of monthly deep learning (LSTM) predictions to conceptual rainfall–runoff (WAPABA model) predictions is performed on almost 500 catchments across Australia with performance results aggregated over a variety of catchment sizes, flow conditions, and hydrological record lengths. The study period covers a wet phase followed by a prolonged drought, introducing challenges for making predictions outside of known conditions – challenges that will intensify as climate change progresses. The results show that LSTM models matched or exceeded WAPABA prediction performance for more than two-thirds of the study catchments, the largest performance gains of LSTM versus WAPABA occurred in large catchments, the LSTMs struggled less to generalise than the WAPABA models (e.g. making predictions under new conditions), and catchments with few training observations due to the monthly time step did not demonstrate a clear benefit with either WAPABA or LSTM.

The Stimulating Factors of Cognitive Memory and the Meaning and Influence of Cognitive Conception

This paper studies the stimulating factors of cognitive memory and the significance and influence of such factors on cognitive concepts. Through a comprehensive review of literature and experimental data, this paper explores how different types of stimuli affect individual memory and cognitive architecture. Through literature research combined with investigation and analysis, the relationship between the themes was further determined. It is found that stimuli can significantly affect the formation of cognitive concepts, thus changing individual learning and decision-making. And further research shows that different kinds of stimuli have a great degree of difference in the change of cognitive structure. Finally, based on this research, this paper analyzes how to solve the existing social problems.

Implementing deep learning-based named entity recognition for obtaining narcotics abuse data in Indonesia

<span lang="EN-US">The availability of drug abuse data from the official website of the National Narcotics Board of Indonesia is not up-to-date. Besides, the drug reports from Indonesian National Narcotics Board are only published once a year. This study aims to utilize online news sites as a data source for collecting information about drug abuse in Indonesia. In addition, this study also builds a named entity recognition (NER) model to extract information from news texts. The primary NER model in this study uses the convolutional neural network-long short-term memory (CNNs-LSTM) architecture because it can produce a good performance and only requires a relatively short computation time. Meanwhile, the baseline NER model uses the bidirectional long short-term memory-conditional random field (Bi-LSTMs-CRF) architecture because it is easy to implement using the Flair framework. The primary model that has been built results in a performance (F1 score) of 82.54%. Meanwhile, the baseline model only results in a performance (F1 score) of 69.67%. Then, the raw data extracted by NER is processed to produce the number of drug suspects in Indonesia from 2018-2020. However, the data that has been produced is not as complete as similar data sourced from Indonesian National Narcotics Board publications.</span>

Using LSTM to translate Thai sign language to text in real time

Between 2019 and 2022, as the Covid-19 pandemic unfolded, numerous countries implemented lockdown policies, leading most corporate companies to permit employees to work from home. Communication and meetings transitioned to online platforms, replacing face-to-face interactions. This shift posed challenges for deaf or hearing-impaired individuals who rely on sign language, using hand gestures for communication. However, it also affected those who can hear clearly but lack knowledge of sign language. Unfortunately, many online meeting platforms lack sign language translation features. This study addresses this issue, focusing on Thai sign language. The objective is to develop a model capable of translating Thai sign language in real-time. The Long Short-Term Memory (LSTM) architecture is employed in conjunction with MediaPipe Holistic for data collection. MediaPipe Holistic captures keypoints of hand, pose, and head, while the LSTM model translates hand gestures into a sequence of words. The model’s efficiency is assessed based on accuracy, with real-time testing achieving an 86% accuracy, slightly lower than the performance on the test dataset. Nonetheless, there is room for improvement, such as expanding the dataset by collecting data from diverse individuals, employing data augmentation techniques, and incorporating an attention mechanism to enhance model accuracy.

An Adaptive Parallel Pattern Based Design for Molecular Dynamic Simulation

In Parallel programming, a programmer needs to understand hardware environment, programming paradigm and primitives available in the programming language. Most of the time, parallel programmes are written for a specific architecture and cannot typically adapt to other architectures Particularly, programs written for shared memory architectures are unsuitable for distributed or hybrid architectures. This paper proposes Adaptive Design Pattern for Parallel Programming to improve adaptability, flexibility with achieving performance on different architectures. Molecular Dynamics (MD) simulation is required to scale to various architectures from simple machine to cluster of workstations. In this study, MD Simulation experimented using both pure benchmark code and code based on adaptive design patterns. Redesigned MD Simulation with Adaptive Design Pattern claims parallel efficiency from 56% to 90% for different number of processing elements used. The solution demonstrates adaptability to different architectures and scalability to use with large number of atoms and long duration simulation.

RNN-BiLSTM-CRF based amalgamated deep learning model for electricity theft detection to secure smart grids.

Electricity theft presents a substantial threat to distributed power networks, leading to non-technical losses (NTLs) that can significantly disrupt grid functionality. As power grids supply centralized electricity to connected consumers, any unauthorized consumption can harm the grids and jeopardize overall power supply quality. Detecting such fraudulent behavior becomes challenging when dealing with extensive data volumes. Smart grids provide a solution by enabling two-way electricity flow, thereby facilitating the detection, analysis, and implementation of new measures to address data flow issues. The key objective is to provide a deep learning-based amalgamated model to detect electricity theft and secure the smart grid. This research introduces an innovative approach to overcome the limitations of current electricity theft detection systems, which predominantly rely on analyzing one-dimensional (1-D) electric data. These approaches often exhibit insufficient accuracy when identifying instances of theft. To address this challenge, the article proposes an ensemble model known as the RNN-BiLSTM-CRF model. This model amalgamates the strengths of recurrent neural network (RNN) and bidirectional long short-term memory (BiLSTM) architectures. Notably, the proposed model harnesses both one-dimensional (1-D) and two-dimensional (2-D) electricity consumption data, thereby enhancing the effectiveness of the theft detection process. The experimental results showcase an impressive accuracy rate of 93.05% in detecting electricity theft, surpassing the performance of existing models in this domain.

The digital preservation of national memorial architecture: A new approach with BIM and VR technology

National memorial architecture serves as a testament to civilization, embodying and perpetuating a nation’s culture, values, and identity. It facilitates the preservation of historical events or memories for future generations to reference while symbolising national pride and unity. However, the conservation of such monuments currently faces numerous challenges, including the impact of unforeseeable forces, excessive tourism, and inadequate funding. In response to these challenges, this paper proposes a solution utilizing Building Information Modeling (BIM) and Virtual Reality (VR) technologies for digital preservation. BIM enables comprehensive data collection of commemorative structures, while VR provides immersive virtual experiences, enhancing public understanding and engagement with these monuments. Using the People’s Heroes Memorial Tower in Zhangcun, Siming Mountain Martyrs Cemetery, Ningbo, Zhejiang, China, as a case study, this paper demonstrates the entire process of digital preservation and explores the potential applications of these technologies in safeguarding national memorial architecture.

An efficient approach based on a novel 1D-LBP for the detection of bearing failures with a hybrid deep learning method

Bearings serve as fundamental components in the transmission of motion for rotating machinery. The occurrence of mechanical wear and subsequent bearing failures within these rotating systems can lead to diminished operational efficiency and, if left unaddressed, may result in the complete cessation of the system's function. Hence, there exists a critical need for effective monitoring methodologies aimed at accurately detecting faults in such systems, preferably in their nascent stages. This study presents a novel approach to fault diagnosis leveraging vibration data obtained from bearings. Initially, a feature extraction technique is devised, which incorporates localized signal variations. Subsequently, these features, extracted via MM-1D-LBP, are utilized in conjunction with a hybrid deep learning network based on Long Short-Term Memory (LSTM) and one-dimensional Convolutional Neural Network (1D-CNN) architectures for diagnostic purposes. To assess the efficacy of the proposed methodology, experiments were conducted on two distinct datasets acquired from real-world bearing assemblies. In the first dataset, the aim was to predict various failure types (Inner Ring, Outer Ring, Ball). In the second dataset, the objective was to estimate defect sizes using bearing vibration signals corresponding to defects of different dimensions (0.15 cm, 0.5 cm, 0.9 cm) under consistent operating conditions. Remarkably high success rates of 99.31 % and 99.65 % were achieved for the two datasets, respectively, thus underscoring the efficacy of the proposed MM-1D-LBP+1D-CNN-LSTM approach. These findings not only demonstrate the feasibility of the proposed method for fault diagnosis in bearing systems but also suggest its potential applicability across diverse signal categories. Ultimately, this research contributes to advancing the state-of-the-art in fault diagnosis methodologies for rotating machinery, offering enhanced accuracy and early detection capabilities.

A low-cost automatic people-counting system at bus stops using Wi-Fi probe requests and deep learning

AbstractCounting people is an important part of people-centric applications, and the increase in the number of IoT devices has allowed the collection of huge amounts of data to facilitate people counting. The present study seeks to provide a novel, low-cost, automatic people-counting system for the use at bus stops, featuring a sniffing device that can capture Wi-Fi probe requests, and overcoming the problem of Media Access Control (MAC) randomization using deep learning. To make manual data collection considerably easier, a “People Counter” app was designed to collect ground truth data in order to train the model with higher accuracy. A user-friendly, operating system-independent dashboard was created to display the most relevant metrics. A two-step methodological approach was followed comprising device choice and data collection; data analysis and algorithm development. For the data analysis, three different approaches were tested, and among these a deep-learning approach using Convolutional Recurrent Neural Network (CRNN) with Long Short-term Memory (LSTM) architecture produced the best results. The optimal deep learning model predicted the number of people at the stop with a mean absolute error of ~ 1.2 persons, which can be considered a good preliminary result, considering that the experiment was done in a very complex open environment. People-counting systems at bus stops can support better bus scheduling, improve the boarding and alighting time of passengers, and aid the planning of integrated multi-modal transport system networks.

Negative capacitance FET based dual-split control 6T-SRAM cell design for energy efficient and robust computing-in memory architectures

A Negative Capacitance Field effet transistor (NCFET) based Dual split control (DSC) 6T-SRAM cell has been designed and explored with Computing-in memory (CiM) architecture for energy efficient demonstration of Deep neural networks (DNN) basic operation such as Input-Weight (Dot) Product. The impact of ferro electric layer thickness (Tfe) on the SRAM cell perfomance metrics such as read noise margin (RNM), write noise margin (WNM) and energy efficiency for read and write operations have been analyzed at supply voltages of 0.3 V and 0.5 V. It has been observed that due to the steep slope characteristics, the NCFET based DSC 6T-SRAM cell design exhibits better RM, WM, and energy efficiency as compared to the baseline CMOS DSC SRAM cell design at VDD = 0.3 V and 0.5 V respectively (with Tfe range of 1 nm to 3 nm). Further, NCFET dual split control scheme for 6T-SRAM cell demonstrate improved read stability and write ability when compared with NCFET 6 T-SRAM cell design along with improved energy efficiency. NCFET based DSC 6T-SRAM CiM cell design has ∼22.77× and 12.41× lower energy consumption compared to the équivalent baseline 40 nm CMOS/baseline SRAM CiM design and ∼ 25.80× and 22.76× lower energy consumption compared to the NCFET based SRAM CiM at VDD = 0.3 V and 0.5 V respectively. NCFETs have improved steep subthreshold slope characteristics at an optimal Tfe value and NCFET SRAM based CiM circuits are expected to have higher noise margins and lower energy consumption compared to the baseline CMOS designs and are effective for NCFET based computing in-memory architectures with reduced read disturb issues in combination with DSC concept.

Application of BiLSTM-CRF model with different embeddings for product name extraction in unstructured Turkish text

Named entity recognition (NER) plays a pivotal role in Natural Language Processing by identifying and classifying entities within textual data. While NER methodologies have seen significant advancements, driven by pretrained word embeddings and deep neural networks, the majority of these studies have focused on text with well-defined grammar and structure. A significant research gap exists concerning NER in informal or unstructured text, where traditional grammar rules and sentence structure are absent. This research addresses this crucial gap by focusing on the detection of product names within unstructured Turkish text. To accomplish this, we propose a deep learning-based NER model which combines a Bidirectional Long Short-Term Memory (BiLSTM) architecture with a Conditional Random Field (CRF) layer, further enhanced by FastText embeddings. To comprehensively evaluate and compare our model’s performance, we explore different embedding approaches, including Word2Vec and Glove, in conjunction with the Bidirectional Long Short-Term Memory and Conditional Random Field (BiLSTM-CRF) model. Furthermore, we conduct comparisons against BERT to assess the efficacy of our approach. Our experimentation utilizes a Turkish e-commerce dataset gathered from the internet, where traditional grammatical and structural rules may not apply. The BiLSTM-CRF model with FastText embeddings achieved an F1 score value of 57.40%, a precision value of 55.78%, and a recall value of 59.12%. These results indicate promising performance in outperforming other baseline techniques. This research contributes to the field of NER by addressing the unique challenges posed by unstructured Turkish text and opens avenues for improved entity recognition in informal language settings, with potential applications across various domains.

H3DM: A High-bandwidth High-capacity Hybrid 3D Memory Design for GPUs

Graphics Processing Units (GPUs) are widely used for modern applications with huge data sizes. However, the performance benefit of GPUs is limited by their memory capacity and bandwidth. Although GPU vendors improve memory capacity and bandwidth using 3D memory technology (HBM), many important workloads with terabytes of data still cannot fit in the provided capacity and are bound by the provided bandwidth. With a limited GPU memory capacity, programmers should handle the data movement between GPU and host memories by themselves, causing a significant programming burden. To improve programming ease, GPUs use a unified address space with the host that allows over-subscribing GPU memory, but this approach is not effective in terms of performance once GPUs encounter memory page faults. Many recent works have tried to remedy capacity and bandwidth bottlenecks using dense non-volatile memories (NVMs) and true-3D stacking. However, these works mainly focus on one bottleneck or do not provide a scalable solution that fits future requirements. In this paper, we investigate true-3D stacking of dense, low-power, and refresh-free non-volatile phase change memory (PCM) on top of state-of-the-art GPU configurations to provide higher capacity and bandwidth within the available area and power budget. The higher density and lower power consumption of PCM provide higher capacity through integrating more cells in each 3D layer and enabling stacking more layers. However, we observe that stacking more than six layers of pure-PCM memory violates the thermal constraint and severely harms the performance and power efficiency due to its higher write latency and energy. Further, it degrades the lifetime of GPU to less than one year. Utilizing a hybrid architecture that leverages the benefits of both DRAM and PCM memories has been widely studied by prior proposals; however, true-3D integration of such a hybrid memory architecture especially on top of state-of-the-art powerful GPU architecture has not been investigated yet. We experimentally demonstrate that by covering 80% of write requests in DRAM and eliminating refresh overhead, true-3D stacking of eight 32GB layers of PCM along with two 8GB layers of DRAM is possible resulting in a total of 272GB memory capacity. Based on the explored design requirements, We propose a 3D high-bandwidth high-capacity hybrid memory (H3DM) system utilizing a hybrid-3D (H3D)-aware remapping scheme to reduce expensive PCM writes to under 20% while avoiding DRAM refresh overhead. H3DM improves the performance up to 291% compared to the baseline GPU architecture while remaining within only 3% of an ideal case with DRAM-like access latency, on average. Moreover, by increasing the dataset size above the baseline GPU memory space, H3DM improves performance and power up to 648% and 87% compared to the baseline GPU architecture since it avoids expensive data transfers through off-chip communication links.

Extension VM: Interleaved Data Layout in Vector Memory

While vector architecture is widely employed in processors for neural networks, signal processing, and high-performance computing; however, its performance is limited by inefficient column-major memory access. The column-major access limitation originates from the unsuitable mapping of multidimensional data structures to two-dimensional vector memory spaces. In addition, the traditional data layout mapping method creates an irreconcilable conflict between row- and column-major accesses. Ideally, both row- and column-major accesses can take advantage of the bank parallelism of vector memory. To this end, we propose the Interleaved Data Layout (IDL) method in vector memory, which can distribute vector elements into different banks regardless of whether they are in the row- or column-major category, so that any vector memory access can benefit from bank parallelism. Additionally, we propose an Extension Vector Memory (EVM) architecture to achieve IDL in vector memory. EVM can support two data layout methods and vector memory access modes simultaneously. The key idea is to continuously distribute the data that needs to be accessed from the main memory to different banks during the loading period. Thus, EVM can provide a larger spatial locality level through careful programming and the extension ISA support. The experimental results showed a 1.43-fold improvement of state-of-the-art vector processors by the proposed architecture, with an area cost of only 1.73%. Furthermore, the energy consumption was reduced by 50.1%.

Deep learning-assisted Hubble parameter analysis

We turn our attention on evaluating the most recent Hubble parameter data measured via the differential evolution of cosmic-chronometers from a deep learning perspective. To achieve this goal, we start our investigation by introducing the selected theoretical setup and compiling the most recent statistical data obtained in cosmology experiments. Then we implement a tuned version of the long-short term memory (LSTM) architecture and run it to predict possible values of the Cosmic Hubble parameter for different red-shift states. Since we observe a good correlation between the observed and predicted datasets of the Hubble parameter, we conclude that the machine learning approaches can play important roles in the future cosmology investigations.

Enhanced transformer long short-term memory framework for datastream prediction

In machine learning, datastream prediction is a challenging issue, particularly when dealing with enormous amounts of continuous data. The dynamic nature of data makes it difficult for traditional models to handle and sustain real-time prediction accuracy. This research uses a multi-processor long short-term memory (MPLSTM) architecture to present a unique framework for datastream regression. By employing several central processing units (CPUs) to divide the datastream into multiple parallel chunks, the MPLSTM framework illustrates the intrinsic parallelism of long short-term memory (LSTM) networks. The MPLSTM framework ensures accurate predictions by skillfully learning and adapting to changing data distributions. Extensive experimental assessments on real-world datasets have demonstrated the clear superiority of the MPLSTM architecture over previous methods. This study uses the transformer, the most recent deep learning breakthrough technology, to demonstrate how well it can handle challenging tasks and emphasizes its critical role as a cutting-edge approach to raising the bar for machine learning.

Space–time recurrent memory network

Transformers have recently been popular for learning and inference in the spatial–temporal domain. However, their performance relies on storing and applying attention to the feature tensor of each frame in video. Hence, their space and time complexity increase linearly as the length of video grows, which could be very costly for long videos. We propose a novel visual memory network architecture for the learning and inference problem in the spatial–temporal domain. We maintain a fixed set of memory slots in our memory network and propose an algorithm based on Gumbel-Softmax to learn an adaptive strategy to update this memory. Finally, this architecture is benchmarked on the video object segmentation (VOS) and video prediction problems. We demonstrate that our memory architecture achieves state-of-the-art results, outperforming transformer-based methods on VOS and other recent methods on video prediction while maintaining constant memory capacity independent of the sequence length.

Study of Ge‐Rich Ge–Sb–Te Device‐Dependent Segregation for Industrial Grade Embedded Phase‐Change Memory

Ge‐rich Ge–Sb–Te (GGST) alloys are the most promising materials for phase‐change memory in embedded applications, being able to fulfill the tough data retention requirements of automotive and consumer markets. GGST alloys are sensitive to thermal budgets and spatial confinement; thus, memory device process integration and architecture can strongly impact their final electrical properties and reliability. Herein, exploiting a statistical methodology capable to extract quantitative metrics for evaluating by‐process segregation, the inhomogeneity of out‐of‐fab GGST material in function of process parameters is studied such as architecture and alloy composition. The present results with the already known source of segregation, namely the back‐end‐of‐line thermal budget, are compared providing a comprehensive description of the main modulating factors of segregation among these different process parameters.

A Deep Learning Method for the Health State Prediction of Lithium-Ion Batteries Based on LUT-Memory and Quantization

The precise determination of the state of health (SOH) of lithium-ion batteries is critical in the domain of battery management systems. The proposed model in this research paper emulates any deep learning or machine learning model by utilizing a Look Up Table (LUT) memory to store all activation inputs and their corresponding outputs. The operation that follows the completion of training is referred to as the LUT memory preparation procedure. This method’s lookup process supplants the inference process entirely and simply. This is achieved by discretizing the input data and features before binarizing them. The term for the aforementioned operation is the LUT inference method. This procedure was evaluated in this study using two distinct neural network architectures: a bidirectional long short-term memory (LSTM) architecture and a standard fully connected neural network (FCNN). It is anticipated that considerably greater efficiency and velocity will be achieved during the inference procedure when the pre-trained deep neural network architecture is inferred directly. The principal aim of this research is to construct a lookup table that effectively establishes correlations between the SOH of lithium-ion batteries and ensures a degree of imprecision that is tolerable. According to the results obtained from the NASA PCoE lithium-ion battery dataset, the proposed methodology exhibits a performance that is largely comparable to that of the initial machine learning models. Utilizing the error assessment metrics RMSE, MAE, and (MAPE), the accuracy of the SOH prediction has been quantitatively evaluated. The indicators mentioned above demonstrate a significant degree of accuracy when predicting SOH.

A hybrid memory architecture supporting fine-grained data migration

A hybrid memory architecture supporting fine-grained data migration

Scalable and Efficient Orchestration of Machine Learning Workloads on DSPs with Multi-level Memory Architecture

Scalable and Efficient Orchestration of Machine Learning Workloads on DSPs with Multi-level Memory Architecture