Computer Memory Research Articles

Improving AI-assisted video editing: Optimized footage analysis through multi-task learning

In recent years, AI-assisted video editing has shown promising applications. Understanding and analyzing camera language accurately is fundamental in video editing, guiding subsequent editing and production processes. However, many existing methods for camera language analysis overlook computational efficiency and deployment requirements in favor of improving classification accuracy. Consequently, they often fail to meet the demands of scenarios with limited computing power, such as mobile devices. To address this challenge, this paper proposes an efficient multi-task camera language analysis pipeline based on shared representations. This approach employs a multi-task learning architecture with hard parameter sharing, enabling different camera language classification tasks to utilize the same low-level feature extraction network, thereby implicitly learning feature representations of the footage. Subsequently, each classification sub-task independently learns the high-level semantic information corresponding to the camera language type. This method significantly reduces computational complexity and memory usage while facilitating efficient deployment on devices with limited computing power. Furthermore, to enhance performance, we introduce a dynamic task priority strategy and a conditional dataset downsampling strategy. The experimental results demonstrate that achieved a comprehensive accuracy surpassing all previous methods. Moreover, training time was reduced by 66.33%, inference cost decreased by 59.85%, and memory usage decreased by 31.95% on the 2-task dataset MovieShots; on the 4-task dataset AVE, training time was reduced by 95.34%, inference cost decreased by 97.23%, and memory usage decreased by 61.21%.

Angle-dependent image-domain least-squares migration through analytical point spread functions

Image-domain least-squares migration (IDLSM) is an established approach to recover high-fidelity seismic images of subsurface reflectors; this is achieved by removing the blurring effects of the Hessian operator in the standard migration approach with the help of so-called point spread functions (PSFs). However, most of the existing IDLSM approaches recover an angle-independent image of the subsurface reflectors, which is not suitable for subsequent amplitude-variation-with-angle (AVA) analysis. To overcome this limitation, we have developed an angle-dependent IDLSM approach, denoted as AD-IDLSM, which can recover a high-fidelity and high-resolution angle-dependent reflectivity image of subsurface reflectors. The problem is formulated here as an angle-dependent image-domain inversion with PSFs computed by means of full-wave Green’s function. More specifically, we derive an analytical expression to compute angle-dependent PSFs by means of a wave-equation-based Kirchhoff migration (WEBKM) engine, where a localization assumption is made in both spatial directions to decrease the computational cost and memory overhead. The amplitude and traveltime of the Green’s functions involved in the WEBKM approach are estimated by the excitation amplitude and excitation time of the full-wave wavefield. The scattering angle is then approximately estimated from the Poynting vector of the excitation-time field. To stabilize the solution of AD-IDLSM, we use a regularization scheme that applies a second derivative along the direction of the reflection angle of angle-domain common-image gathers (ADCIGs) to ensure continuity in the amplitude variations versus angle and suppress migration artifacts. We demonstrate the effectiveness of the AD-IDLSM approach through two synthetic and one field marine data set; the presented results confirm that AD-IDLSM can create ADCIGs with higher spatial resolution, better amplitude fidelity, and fewer migration artifacts compared with those obtained by its migration counterpart. Moreover, AD-IDLSM amplitude variations with angle are shown to closely resemble the theoretical AVA curve of the reflectors.

Efficient large-scale 3D gravity modeling using a fast evaluate kernel matrix combined with compressed matrix techniques

A fast calculated kernel matrix method is coupled with a compressed matrix technique to solve the large-scale gravity forward-modeling problem. This method accelerates the coefficient matrix computation by reducing the arctangent, logarithm, and multiplication functions in the prismatic gravity analytical expression. In addition, the use of the compressed matrix technique presents a significant advantage in that it does not require the storage of redundant kernel matrices, further reducing the memory requirements and computation time. Moreover, the discrete convolution of the compressed matrix with density is executed through the 2D fast Fourier transform (FFT). Two typical synthetic models are used to test the performance of the novel algorithm. The results demonstrate that the developed algorithm is approximately 15 times faster than the traditional algorithm. Concurrently, it demands almost 1/7th of the memory while ensuring equivalent computational accuracy. To further illustrate the capability of the algorithm, we apply our method to the terrain correction of an airborne gravity data set using a real digital elevation model. Our approach efficiently calculates 250,000 observation points across 25 million cells in only 5.42 s, compared with 234.81 s for the latest 2D Gauss-FFT method.

A novel weight index-based uniform partition technique of multi-dimensional probability space for structural uncertainty quantification

Accurately and efficiently achieving the uncertainty quantification of engineering structures is a challenging issue. The direct probability integral method (DPIM) provides an effective pathway to address this issue. However, the key partition technique via Voronoi cell of DPIM requires a prohibitive computational burden for multi-dimensional probability space. Moreover, due to the distributed nonuniformity of representative points, the accuracy of DPIM with the partition technique via Voronoi cell (DPIM-Voronoi) for obtaining the response probability density function (PDF) of structures with multi-dimensional probability space still needs to be improved. To this end, a novel weight index-based uniform partition technique is proposed in this study. This technique can generate uniformly distributed representative points and calculate their assigned probabilities using the weight indexes of representative regions. This feature ensures that the representative points can fill the probability space in a highly uniform manner, and avoid the resource-consuming calculation process of assigned probability by Monte Carlo simulation in the original partition technique via Voronoi cell. Based on the proposed technique, DPIM with a Weight index-based Uniform partition technique (DPIM-WU) is developed. Compared to DPIM-Voronoi, the advantages of DPIM-WU include: (1) improving the computational accuracy of response PDF for structures with multi-dimensional probability space, especially in the tail region, leading to improved accuracy of the dynamic reliability; (2) remarkably reducing the computational cost, with minimal computer memory required for the partition process of multi-dimensional probability space; (3) enhancing the robustness to the number of representative points. These advantages are verified through the stochastic response and dynamic reliability analyses of four typical examples, including the 5-story buildings, dry friction system, cylinder structure, and adjacent buildings with pounding motion. Notably, in the stochastic pounding response analysis of adjacent buildings, a stochastic P-bifurcation occurs as the coefficient of variation of the structural parameters decreases.

Integrated in-memory sensor and computing of artificial vision system based on reversible bonding transition-induced nitrogen-doped carbon quantum dots (N-CQDs)

Integrated in-memory sensor and computing of artificial vision system based on reversible bonding transition-induced nitrogen-doped carbon quantum dots (N-CQDs)

CyberCortex.AI: An AI‐based operating system for autonomous robotics and complex automation

AbstractThe underlying framework for controlling autonomous robots and complex automation applications is Operating Systems (OS) capable of scheduling perception‐and‐control tasks, as well as providing real‐time data communication to other robotic peers and remote cloud computers. In this paper, we introduce CyberCortex.AI, a robotics OS designed to enable heterogeneous AI‐based robotics and complex automation applications. CyberCortex.AI is a decentralized distributed OS which enables robots to talk to each other, as well as to High Performance Computers (HPC) in the cloud. Sensory and control data from the robots is streamed toward HPC systems with the purpose of training AI algorithms, which are afterwards deployed on the robots. Each functionality of a robot (e.g., sensory data acquisition, path planning, motion control, etc.) is executed within a so‐called DataBlock of Filters shared through the internet, where each filter is computed either locally on the robot itself or remotely on a different robotic system. The data is stored and accessed via a so‐called Temporal Addressable Memory (TAM), which acts as a gateway between each filter's input and output. CyberCortex.AI has two main components: (i) the CyberCortex.AI.inference system, which is a real‐time implementation of the DataBlock running on the robots' embedded hardware, and (ii) the CyberCortex.AI.dojo, which runs on an HPC computer in the cloud, and it is used to design, train and deploy AI algorithms. We present a quantitative and qualitative performance analysis of the proposed approach using two collaborative robotics applications: (i) a forest fires prevention system based on an Unitree A1 legged robot and an Anafi Parrot 4K drone, as well as (ii) an autonomous driving system which uses CyberCortex.AI for collaborative perception and motion control.

Spin valve effect in the van der Waals heterojunction of Fe3GeTe2/tellurene/Fe3GeTe2

Spintronic devices are regarded as prime candidates for addressing the demands of emergent applications such as in-memory computing and the Internet of Things, characterized by requirements for high speed, low energy consumption, and elevated storage density. Among these, spin valves, serving as fundamental structures of magnetic random-access memory, have garnered substantial attention in recent years. This study introduces an all van der Waals (vdW) heterostructure composed of Fe3GeTe2 (FGT)/tellurene/FGT, wherein a thin layer of Weyl semiconductor Te is interposed between two ferromagnetic FGT layers. The proposed configuration exhibits a characteristic spin valve effect at temperatures below 160 K. This effect is attributed to spin-dependent transport and spin-dependent scattering phenomena occurring at the interfaces of the constituent materials. Furthermore, as temperature decreases, the magnetoresistance ratio (MR) of the device increases, indicative of the heightened polarization ratio of FGT, with an MR of 0.43% achievable as the temperature approaches 5 K. This investigation elucidates the underlying operational mechanisms of two-dimensional spin valve devices and lays the groundwork for the realization of spin-based integrated circuits.

Real-time identification of noise type contaminated in surface electromyogram signals using efficient statistical features

Different types of noise contaminating the surface electromyogram (EMG) signal may degrade the recognition performance. For noise removal, the type of noise has to first be identified. In this paper, we propose a real-time efficient system for identifying a clean EMG signal and noisy EMG signals contaminated with any one of the following three types of noise: electrocardiogram interference, spike noise, and power line interference. Two statistical descriptors, kurtosis and skewness, are used as input features for the cascading quadratic discriminant analysis classifier. An efficient simplification of kurtosis and skewness calculations that can reduce computation time and memory storage is proposed. The experimental results from the real-time system based on an ATmega 2560 microcontroller demonstrate that the kurtosis and skewness values show root mean square errors between the traditional and proposed efficient techniques of 0.08 and 0.09, respectively. The identification accuracy with five-fold cross-validation resulting from the quadratic discriminant analysis classifier is 96.00%.

Neuromorphic learning and recognition in WO3−x thin film-based forming-free flexible electronic synapses

In pursuing advanced neuromorphic applications, this study introduces the successful engineering of a flexible electronic synapse based on WO3-x, structured as W/WO3-x/Pt/Muscovite-Mica. This artificial synapse is designed to emulate crucial learning behaviors fundamental to in-memory computing. We systematically explore synaptic plasticity dynamics by implementing pulse measurements capturing potentiation and depression traits akin to biological synapses under flat and different bending conditions, thereby highlighting its potential suitability for flexible electronic applications. The findings demonstrate that the memristor accurately replicates essential properties of biological synapses, including short-term plasticity (STP), long-term plasticity (LTP), and the intriguing transition from STP to LTP. Furthermore, other variables are investigated, such as paired-pulse facilitation, spike rate-dependent plasticity, spike time-dependent plasticity, pulse duration-dependent plasticity, and pulse amplitude-dependent plasticity. Utilizing data from flat and differently bent synapses, neural network simulations for pattern recognition tasks using the Modified National Institute of Standards and Technology dataset reveal a high recognition accuracy of ∼95% with a fast learning speed that requires only 15 epochs to reach saturation.

Elastic reverse time migration using an efficient and accurate adaptive variable grid discretization method

Abstract The conventional reverse time migration utilizes regularly sampled computational grids to simulate wave propagation. Selecting the appropriate grid sampling is important for computational accuracy and efficiency. In general, the uniform-size grid cannot represent the complexity of the geology well. The grid may appear sparse in the low-velocity zone, especially in shallow depths where dispersion may occur. Conversely, it may appear excessively dense in the high-velocity zone, such as at greater depths or within a salt body, which results in higher computational memory and time consumption. To overcome these issues, we developed an efficient and accurate adaptive variable grid discretization method that automatically selects the vertical grid size based on the velocity, depth, and dominant frequency of the wavelet in elastic medium. Then we reformulated the elastic equations based on the adaptive variable grid by introducing a mapping relationship. To test the effectiveness, accuracy, and efficiency of the equation, we implemented it to both the forward propagation and migration of elastic wavefield. Synthetic numerical examples demonstrate that our proposed method can achieve elastic wavefield separation and no significant dispersion phenomenon. The multi-component imaging accuracy of reverse time migration is nearly equivalent to the traditional method, while significantly improving computational efficiency and saving storage space.

Bi-Directional and Operand-Controllable In-Memory Computing for Boolean Logic and Search Operations with Row and Column Directional SRAM (RC-SRAM).

The von Neumann architecture is no longer sufficient for handling large-scale data. In-memory computing has emerged as the potent method for breaking through the memory bottleneck. A new 10T SRAM bitcell with row and column control lines called RC-SRAM is proposed in this article. The architecture based on RC-SRAM can achieve bi-directional and operand-controllable logic-in-memory and search operations through different signal configurations, which can comprehensively respond to various occasions and needs. Moreover, we propose threshold-controlled logic gates for sensing, which effectively reduces the circuit area and improves accuracy. We validate the RC-SRAM with a 28 nm CMOS technology, and the results show that the circuits are not only full featured and flexible for customization but also have a significant increase in the working frequency. At VDD = 0.9 V and T = 25 °C, the bi-directional search frequency is up to 775 MHz and 567 MHz, and the speeds for row and column Boolean logic reach 759 MHz and 683 MHz.

Isosurface-based marching cube algorithm for smooth geometric topology optimization within adaptive octree SBFE approach

In the era of Industry 4.0, the prominence of 3D printing as a pivotal manufacturing technology has greatly expanded, particularly within the domain of additive manufacturing (AM). Among the thriving research applications tailored for integration with AM, topology optimization (TO) has emerged as a resounding success. Given the prerequisite of TO for high-resolution meshing to ensure visual clarity in result depiction, researchers have been consistently driven to develop advanced techniques to refine optimal designs, thus elevating the challenge and popularity within this research realm. This paper presents a novel approach integrating an adaptive image-based octree mesh scaled boundary finite element (SBFE) framework with an evolutionary methodology that can effectively address the persistent challenges inherent to TO. A novel hierarchical SBFE mesh analysis not only facilitates efficient and precise TO but also substantially reduces computational resource demands. Furthermore, the pre-conditioned conjugated gradient (PCG) method is adopted to process practical-scale problems, minimizing computer memory resources. Additionally, the proposed work incorporates a post-processing technique utilizing the isosurface function based on a marching cube algorithm, thereby smoothing the boundaries of optimal results. Consequently, this research extends the horizons of design possibilities, particularly in the creation of intricate 3D structures, which can be seamlessly realized through additive manufacturing and 3D printing.

Fast and robust analog in-memory deep neural network training

Analog in-memory computing is a promising future technology for efficiently accelerating deep learning networks. While using in-memory computing to accelerate the inference phase has been studied extensively, accelerating the training phase has received less attention, despite its arguably much larger compute demand to accelerate. While some analog in-memory training algorithms have been suggested, they either invoke significant amount of auxiliary digital compute—accumulating the gradient in digital floating point precision, limiting the potential speed-up—or suffer from the need for near perfectly programming reference conductance values to establish an algorithmic zero point. Here, we propose two improved algorithms for in-memory training, that retain the same fast runtime complexity while resolving the requirement of a precise zero point. We further investigate the limits of the algorithms in terms of conductance noise, symmetry, retention, and endurance which narrow down possible device material choices adequate for fast and robust in-memory deep neural network training.

A novel framework for developing a machine learning-based forecasting model using multi-stage sensitivity analysis to predict the energy consumption of PCM-integrated building

Accurate machine learning (ML) predictions for the early stages of the building design are crucial to construct energy-efficient buildings utilizing limited resources. Several studies have employed ML methods for energy consumption (EC) prediction without considering the utmost crucial PCM-integrated building design parameters. In addition, reducing the dataset by considering only the most significant building design parameters before applying the ML-based method would be beneficial for reducing the computational power and memory usage of the system as well as utilizing less time in the modelling process. To this end, this research presents a novel framework to establish the most robust and reliable ML-based prediction model with less complexity, considering only the most influential PCM-integrated building design parameters. These parameters were identified for future scenarios of hot semi-arid (BSh) climate zones using multi-stage sensitivity analysis. Afterward, a reduced EC database based on the most significant building's early-design-stage parameters (EDSPs) was utilized to formulate several multi-expression programming (MEP) and support vector machines (SVM)-based forecasting models, considering the variations in their hyperparameter values. Formulated prediction models have shown less time utilization through the training and testing phases for the EC evaluations of selected PCM-integrated building compared to the physical-modelling process. Several statistical parameters were used to test and validate the performance of the formulated prediction models. The acquired model evaluation and validation results demonstrated that the MEP-based prediction model (MEP15) exhibited the highest level of reliability and accuracy, showing an R2 value of >95% for both the training and testing phases. The model's interpretability showed that, throughout the parametric analysis, the developed prediction model adhered to the system's physical boundary constraints. Also, the best-performing prediction model showed energy savings of up to 12% for building integrated with PCM having a melting temperature of 28 °C. Conclusively, this research demonstrated that the developed MEP-based prediction model could be employed to precisely forecast the EC for selected PCM-incorporated building in the whole BSh climate, considering important EDSPs while utilizing minimal resources.

Semantic memory-based dynamic neural network using memristive ternary CIM and CAM for 2D and 3D vision.

The brain is dynamic, associative, and efficient. It reconfigures by associating the inputs with past experiences, with fused memory and processing. In contrast, AI models are static, unable to associate inputs with past experiences, and run on digital computers with physically separated memory and processing. We propose a hardware-software co-design, a semantic memory-based dynamic neural network using a memristor. The network associates incoming data with the past experience stored as semantic vectors. The network and the semantic memory are physically implemented on noise-robust ternary memristor-based computing-in-memory (CIM) and content-addressable memory (CAM) circuits, respectively. We validate our co-designs, using a 40-nm memristor macro, on ResNet and PointNet++ for classifying images and three-dimensional points from the MNIST and ModelNet datasets, which achieves not only accuracy on par with software but also a 48.1 and 15.9% reduction in computational budget. Moreover, it delivers a 77.6 and 93.3% reduction in energy consumption.

Hafnium Oxide-Based Ferroelectric Devices for In-Memory Computing: Resistive and Capacitive Approaches

Hafnium Oxide-Based Ferroelectric Devices for In-Memory Computing: Resistive and Capacitive Approaches

Double-Ended Bit-Stealing for Algebraic Data Types

Algebraic data types are central to the development and evaluation of most functional programs. It is therefore important for compilers to choose compact and efficient representations of such types, in particular to achieve good memory footprints for applications. Algebraic data types are most often represented using blocks of memory where the first word is used as a so-called tag, carrying information about the constructor, and the following words are used for carrying the constructor's arguments. As an optimisation, lists are usually represented more compactly, using a technique called bit-stealing, which, in its simplest form, uses the word-alignment property of pointers to byte-addressed allocated memory to discriminate between the nil constructor (often represented as 0x1) and the cons constructor (aligned pointer to allocated pair). Because the representation supports that all values can be held uniformly in one machine word, possibly pointing to blocks of memory, type erasure is upheld. However, on today's 64-bit architectures, memory addresses (pointers) are represented using only a subset of the 64 bits available in a machine word, which leave many bits unused. In this paper, we explore the use, not only of the least-significant bits of pointers, but also of the most-significant bits, for representing algebraic data types in a full ML compiler. It turns out that, with such a particular utilisation of otherwise unused bits, which we call double-ended bit-stealing, it is possible to choose unboxed representations for a large set of data types, while still not violating the principle of uniform data-representations. Examples include Patricia trees, union-find data structures, stream data types, internal language representations for types and expressions, and mutually recursive ASTs for full language definitions. The double-ended bit-stealing technique is implemented in the MLKit compiler and speedup ranges from 0 to 26 percent on benchmarks that are influenced by the technique. For MLKit, which uses abstract data types extensively, compilation speedups of around 9 percent are achieved for compiling MLton (another Standard ML compiler) and for compiling MLKit itself.

Enhancing internet of things security using entropy-informed RF-DNA fingerprint learning from Gabor-based images

Internet of Things (IoT) deployments are anticipated to reach 29.42 billion by the end of 2030 at an average growth rate of 16% over the next 6 years. These deployments represent an overall growth of 201.4% in operational IoT devices from 2020 to 2030. This growth is alarming because IoT devices have permeated all aspects of our daily lives, and most lack adequate security. IoT-connected systems and infrastructures can be secured using device identification and authentication, two effective identity-based access control mechanisms. Physical Layer Security (PLS) is an alternative or augmentation to cryptographic and other higher-layer security schemes often used for device identification and authentication. PLS does not compromise spectral and energy efficiency or reduce throughput. Specific Emitter Identification (SEI) is a PLS scheme capable of uniquely identifying senders by passively learning emitter-specific features unintentionally imparted on the signals during their formation and transmission by the sender’s radio frequency (RF) front end. This work focuses on image-based SEI because it produces deep learning (DL) models that are less sensitive to external factors and better generalize to different operating conditions. More specifically, this work focuses on reducing the computational cost and memory requirements of image-based SEI with little to no reduction in performance by selecting the most informative portions of each image using entropy. These image portions or tiles reduce memory storage requirements by 92.8% and the DL training time by 81% while achieving an average percent correct classification performance of 91% and higher for SNR values of 15 dB and higher with individual emitter performance no lower than 87.7% at the same SNR. Compared with another state-of-the-art time-frequency (TF)-based SEI approach, our approach results in superior performance for all investigated signal-to-noise ratio conditions, the largest improvement being 21.7% at 9 dB and requires 43% less data.

Efficient Numerical Implementation of the Time-Fractional Stochastic Stokes–Darcy Model

This paper presents an efficient numerical method for the fractional-order generalization of the stochastic Stokes–Darcy model, which finds application in various engineering, biomedical and environmental problems involving interaction between free fluid flow and flows in porous media. Unlike the classical model, this model allows taking into account the hereditary properties of the process under uncertainty conditions. The proposed numerical method is based on the combined use of the sparse grid stochastic collocation method, finite element/finite difference discretization, a fast numerical algorithm for computing the Caputo fractional derivative, and a cost-effective ensemble strategy. The hydraulic conductivity tensor is assumed to be uncertain in this problem, which is modeled by the reduced Karhunen–Loève expansion. The stability and convergence of the deterministic numerical method have been rigorously proved and validated by numerical tests. Utilizing the ensemble strategy allowed us to solve the deterministic problem once for all samples of the hydraulic conductivity tensor, rather than solving it separately for each sample. The use of the algorithm for computing the fractional derivatives significantly reduced both computational cost and memory usage. This study also analyzes the influence of fractional derivatives on the fluid flow process within the fractional-order Stokes–Darcy model under uncertainty conditions.

Energy-efficient synthetic antiferromagnetic skyrmion-based artificial neuronal device

Magnetic skyrmions offer unique characteristics such as nanoscale size, particle-like behavior, topological stability, and low depinning current density. These properties make them promising candidates for next-generation spintronics-based memory and neuromorphic computing. However, one of their distinctive features is their tendency to deviate from the direction of the applied driving force that may lead to the skyrmion annihilation at the edge of nanotrack during skyrmion motion, known as the skyrmion Hall effect (SkHE). To overcome this problem, synthetic antiferromagnetic (SAF) skyrmions that having bilayer coupling effect allows them to follow a straight path by nullifying SkHE making them alternative for ferromagnetic (FM) counterpart. This study proposes an integrate-and-fire (IF) artificial neuron model based on SAF skyrmions with asymmetric wedge-shaped nanotrack having self-sustainability of skyrmion numbers at the device window. The model leverages inter-skyrmion repulsion to replicate the IF mechanism of biological neuron. The device threshold, determined by the maximum number of pinned skyrmions at the device window, can be adjusted by tuning the current density applied to the nanotrack. Neuronal spikes occur when initial skyrmion reaches the detection unit after surpassing the device window by the accumulation of repulsive force that result in reduction of the device’s contriving current results to design of high energy efficient for neuromorphic computing. Furthermore, work implements a binarized neuronal network accelerator using proposed IF neuron and SAF-SOT-MRAM based synaptic devices for national institute of standards and technology database image classification. The presented approach achieves significantly higher energy efficiency compared to existing technologies like SRAM and STT-MRAM, with improvements of 2.31x and 1.36x, respectively. The presented accelerator achieves 1.42x and 1.07x higher throughput efficiency per Watt as compared to conventional SRAM and STT-MRAM based designs.