Computational Memory Requirements Research Articles

Interference Analysis of Very Fast Transient Overvoltages in Gas-Insulated Substations Using a Hybrid Algorithm

The reliable and efficient computational approach for the cable interference in gas insulated substations (GIS) is presented. The hybrid domain decomposition method-multilevel fast multi-pole algorithm (DDM-MLFMA) is presented herein. This technique determines the specifications of GIS cables for RG58 and AWG23. Self and mutual interference are identified utilizing the very fast transient overvoltage (VFTO) transient interference signal. The hybrid technique offers a numerical simulation to replicate the impact of the VFTO interference signal. Utilizing the matrix-vector multiplication product (MVX) to address compression and approximation challenges, the hybrid approach proves reliable and pragmatic. Because of its computational nature and explicit factorization reduction, this strategy reduces computation time and memory requirements. Consequently, the system's complexity follows a linear trend under this proposed approach. The computed results are juxtaposed with traditional methodologies to validate the effectiveness of the proposed algorithm.

FPGA-QNN: Quantized Neural Network Hardware Acceleration on FPGAs

Recently, convolutional neural networks (CNNs) have received a massive amount of interest due to their ability to achieve high accuracy in various artificial intelligence tasks. With the development of complex CNN models, a significant drawback is their high computational burden and memory requirements. The performance of a typical CNN model can be enhanced by the improvement of hardware accelerators. Practical implementations on field-programmable gate arrays (FPGA) have the potential to reduce resource utilization while maintaining low power consumption. Nevertheless, when implementing complex CNN models on FPGAs, these may may require further computational and memory capacities, exceeding the available capacity provided by many current FPGAs. An effective solution to this issue is to use quantized neural network (QNN) models to remove the burden of full-precision weights and activations. This article proposes an accelerator design framework for FPGAs, called FPGA-QNN, with a particular value in reducing high computational burden and memory requirements when implementing CNNs. To approach this goal, FPGA-QNN exploits the basics of quantized neural network (QNN) models by converting the high burden of full-precision weights and activations into integer operations. The FPGA-QNN framework comes up with 12 accelerators based on multi-layer perceptron (MLP) and LeNet CNN models, each of which is associated with a specific combination of quantization and folding. The outputs from the performance evaluations on Xilinx PYNQ Z1 development board proved the superiority of FPGA-QNN in terms of resource utilization and energy efficiency in comparison to several recent approaches. The proposed MLP model classified the FashionMNIST dataset at a speed of 953 kFPS with 1019 GOPs while consuming 2.05 W.

Correlation-based pruning algorithm with weight compensation for feedforward neural networks

Abstract Optimizing neural network architectures through effective pruning techniques has become essential to balancing model complexity and accuracy. This study introduces a novel correlation-based approach to systematically reduce network size by identifying and removing redundant neurons based on their activation correlations. By selectively pruning neurons while compensating for their contributions, the method maintains model fidelity across diverse datasets. Results demonstrate substantial architecture reductions with minimal performance impact: For the MNIST dataset, the number of neurons in hidden layers was reduced from 128-128 to 118-93, while maintaining a high accuracy of 97.59%. Comparative analysis indicates that this pruning approach achieves competitive or superior results compared to state-of-the-art methods while reducing computational complexity and memory requirements by up to 25%. The findings highlight the potential of correlation-driven pruning strategies to optimize neural networks, making them more efficient and adaptable to resource-constrained environments.

D-SELD: Dataset-Scalable Exemplar LCA-Decoder

Abstract Neuromorphic computing has recently gained significant attention as a promising approach for developing energy-efficient, massively parallel computing systems inspired by the spiking behavior of the human brain and natively mapping spiking neural networks (SNNs). Effective training algorithms for SNNs are imperative for increased adoption of neuromorphic platforms; however, SNN training continues to lag behind advances in other classes of ANN. In this paper, we reduce this gap by proposing an innovative encoder–decoder technique that leverages sparse coding and the locally competitive algorithm (LCA) to provide an algorithm specifically designed for neuromorphic platforms. Using our proposed Dataset-Scalable Exemplar LCA-Decoder we reduce the computational demands and memory requirements associated with training SNNs using error backpropagation methods on increasingly larger training sets. We offer a solution that can be scalably applied to datasets of any size. Our results show the highest reported top-1 test accuracy using SNNs on the ImageNet and CIFAR100 datasets, surpassing previous benchmarks. Specifically, we achieved a record top-1 accuracy of 80.75% on ImageNet (ILSVRC2012 validation set) and 79.32% on CIFAR100 using SNNs.

An Efficient Lane Detection Network with Channel-Enhanced Coordinate Attention

Lane detection based on semantic segmentation can achieve high accuracy, but, in recent years, it does not have a mobile-friendly cost, which is caused by the complex iteration and costly convolutions in convolutional neural networks (CNNs) and state-of-the-art (SOTA) models based on CNNs, such as spatial CNNs (SCNNs). Although the SCNN has shown its capacity to capture the spatial relationships of pixels across rows and columns of an image, the computational cost and memory requirement needed cannot be afforded with mobile lane detection. Inspired by the channel attention and self-attention machine, we propose an integrated coordinate attention (ICA) module to capture the spatial-wise relationships of pixels. Furthermore, due to the lack of enhancement in the channel dimension, we created an efficient network with a channel-enhanced coordinate attention block named CCA, composed of ICA and other channel attention modules, for all-dimension feature enhancement. As a result, by replacing many repeated or iterative convolutions with the attention mechanism, CCA reduces the computational complexity. Thus, our method achieves a balance of accuracy and speed and has better performance on two lane datasets—TuSimple and ILane. At less than a few tenths of the computational cost, our CCA achieves superior accuracy compared to the SCNN. These results show that the low cost and great performance of our design enable the use of the lane detection task in autopilot scenarios.

Adaptive Factored Incomplete Inverse Matrices

ABSTRACTThe solution of large sparse linear systems is an essential part in many scientific fields. Numerical solution of such systems is usually performed using iterative methods in conjunction with effective preconditioning schemes. A new class of factored approximate inverses is proposed, namely adaptive factored incomplete inverse matrices, which are computed using a recursive Schur complement‐based approach. This class of approximate inverses does not require the knowledge of a sparsity pattern, which is formed adaptively during computation. For this to be possible, a flexible sparse storage scheme was designed. Several numerical dropping strategies are showcased and discussed, along with the effects of reordering to the preconditioner density and the corresponding convergence behavior. Dropping based on monitoring the growth of elements in the incomplete LU factorization is also presented and discussed along with improvements during computation of the successive Schur complements. A static sparsity pattern variant is also provided and discussed. Implementation details and analysis, for computing the proposed scheme, are given along with the computational complexity and memory requirements. Numerical results depicting the effectiveness and applicability of the proposed scheme are also presented.

A novel tucker decomposition driven taxi travel demand forecasting algorithm

<p>This research aims to enhance taxi travel demand forecasting for sustainable urban traffic management and planning. We extend the Seasonal Autoregressive Integrated Moving Average model into a high-dimensional tensor form by treating the urban transport network as a Euclidean space and introducing Tucker decomposition. This novel approach, both theoretically significant and practically applicable, represents time series data as tensors to better capture multimodal structures and correlations, improving predictive accuracy. Tucker decomposition reduces computational complexity and memory requirements, making it ideal for large-scale urban traffic network prediction. Experiments on six real-world datasets show that the model’s MAE and RMSE are reduced by about 39.43% and 27.01% on average, respectively, compared to the baseline model. Notably, the model is computationally very efficient and takes only a relatively short time to train., suitable for real-time traffic management, congestion mitigation, and resource optimization. In summary, this work innovates time series analysis, providing an efficient and precise tool for urban traffic management and planning, contributing to sustainable urban transportation advancement.<strong></strong></p>

RaNAS: Resource-Aware Neural Architecture Search for Edge Computing

Neural architecture search (NAS) for edge devices is often time-consuming because of long-latency deploying and testing on edge devices. The ability to accurately predict the computation cost and memory requirement for convolutional neural networks (CNNs) in advance holds substantial value. Existing work primarily relies on analytical models, which can result in high prediction errors. This paper proposes a resource-aware NAS (RaNAS) model based on various features. Additionally, a new graph neural network is introduced to predict inference latency and maximum memory requirements for CNNs on edge devices. Experimental results show that, within the error bound of ±1%, RaNAS achieves an accuracy improvement of approximately 8% for inference latency prediction and about 25% for maximum memory occupancy prediction over the state-of-the-art approaches.

Machine learning-based prediction of FeNi nanoparticle magnetization

This work proposes a computationally efficient approach for estimating the magnetization of Fe0.7Ni0.3 body-centered cubic (bcc) nanoparticles (NPs) at room temperature using machine-learning algorithms, in terms of the average magnetic moment per atom, ⟨μ⟩. The magnetization data of isolated NPs were generated using atomistic spin dynamics (ASD) simulations for various nanoparticle shapes (cubes, spheres, octahedra, cones, cylinders, ellipsoids, flakes, and pyramids, with or without nanovoids) and FeNi distributions (random, core-shell, onion, sandwich, and Janus with different boundary planes). More than 1600 NPs were created and split into training and testing sets (70%–30% split), with features including the number of Ni/Fe surface and core atoms, potential energy distributions, pair correlation functions, and coordination distributions. Several machine-learning algorithms, including Random Forest (RF), Elastic Net, Support Vector Regression (SVR), and Gradient Boosting Regression (CatBoost), were applied to predict the average magnetic moment per atom of these NPs. The best-performing models, CatBoost and RF, achieved R2 scores of up to 0.86, demonstrating their accuracy in predicting NP magnetization. Feature analysis highlighted the significance of the interface between Fe and Ni clusters, Fe–Fe interactions, and the presence of Fe on the surface as critical contributors to overall magnetization. Random alloy spherical NPs without porosity exhibited the highest ⟨μ⟩ ∼ 1.6μB due to reduced Ni–Ni interactions. Applying machine-learning methods significantly reduces computational time and memory requirements compared to traditional ASD simulations. This allows for rapid prediction of NPs with desired magnetic properties, making them suitable for various technological applications.

LDF-BNN: A Real-Time and High-Accuracy Binary Neural Network Accelerator Based on the Improved BNext.

Significant progress has been made in industrial defect detection due to the powerful feature extraction capabilities of deep neural networks (DNNs). However, the high computational cost and memory requirement of DNNs pose a great challenge to the deployment of industrial edge-side devices. Although traditional binary neural networks (BNNs) have the advantages of small storage space requirements, high parallel computing capability, and low power consumption, the problem of significant accuracy degradation cannot be ignored. To tackle these challenges, this paper constructs a BNN with layered data fusion mechanism (LDF-BNN) based on BNext. By introducing the above mechanism, it strives to minimize the bandwidth pressure while reducing the loss of accuracy. Furthermore, we have designed an efficient hardware accelerator architecture based on this mechanism, enhancing the performance of high-accuracy BNN models with complex network structures. Additionally, the introduction of multi-storage parallelism alleviates the limitations imposed by the internal transfer rate, thus improving the overall computational efficiency. The experimental results show that our proposed LDF-BNN outperforms other methods in the comprehensive comparison, achieving a high accuracy of 72.23%, an image processing rate of 72.6 frames per second (FPS), and 1826 giga operations per second (GOPs) on the ImageNet dataset. Meanwhile, LDF-BNN can also be well applied to defect detection dataset Mixed WM-38, achieving a high accuracy of 98.70%.

TS‐Mixer: A lightweight text representation model based on context awareness

AbstractLarge pre‐trained models (PTMs) have shown their powerful ability in multiple natural language processing tasks. However, using them in practical application remains a challenge due to the significant computational cost and memory requirements. In order to achieve the balance of computational cost and accuracy, MLP architecture can be used as an alternative to the self‐attention module, such as pNLP‐Mixer and Hyper‐Mixer. Experiments indicate that, MLP‐based models can attain competitive performance with low cost. They maintain the balance of computation cost and accuracy successfully, yet, this is at the expense of not being able to capture short‐range dependencies. In this paper, a novel MLP‐based model, termed TS‐Mixer, is proposed which can capture local dependencies by shifting operation. Compared with other MLP‐based models, the parameters of TS‐Mixer are decoupled from the sequence length, hence it has a smaller model size in long sequence tasks. In addition, TS‐Mixer has linear computational complexity, therefore it can be used as a lightweight alternative to the self‐attention model. Experiments show that the TS‐Mixer outperforms other MLP‐based models, which achieves higher accuracy with fewer parameters in multiple downstream tasks. Notably, compared with pre‐trained models, TS‐Mixer can reach more than 90% of their accuracy with 1% or even one thousandth of the parameters (0.174 ~ 1.2 M). These results demonstrate that TS‐Mixer can achieve a better balance between the computing resources and accuracy. Code is available at: https://github.com/wyl-privacy-project/TS-Mixer.

Exploring the potential of large language models for author profiling tasks in digital text forensics

The rapid advancement of large language models (LLMs) has opened up new possibilities for various natural language processing tasks. This study explores the potential of LLMs for author profiling in digital text forensics, which involves identifying characteristics such as age and gender from writing style—a crucial task in forensic investigations of anonymous or pseudonymous communications. Experiments were conducted using state-of-the-art LLMs, including Polyglot, EEVE, and Bllossom, to evaluate their performance in author profiling. Different fine-tuning strategies, such as full fine-tuning, Low-Rank Adaptation (LoRA), and Quantized LoRA (QLoRA), were compared to determine the most effective methods for adapting LLMs to the specific needs of this task. The results show that fine-tuned LLMs can effectively predict authors’ age and gender based on their writing styles, with Polyglot-based models generally outperforming EEVE and Bllossom models. Additionally, LoRA and QLoRA strategies significantly reduce computational costs and memory requirements while maintaining performance comparable to full fine-tuning. However, error analysis reveals limitations in the current LLM-based approach, including difficulty in capturing subtle linguistic variations across age groups and potential biases from pre-training data. These challenges are discussed and future research directions to address them are proposed. This study underscores the potential of LLMs in author profiling for digital text forensics, suggesting promising avenues for further exploration and refinement.

Quantized kernel recursive q-Rényi-like algorithm

This paper introduces the kernel recursive q-Rényi-like (KRqRL) algorithm, based on the q-Rényi kernel function and the kernel recursive least squares (KRLS) algorithm. To reduce the computational complexity and memory requirements of the KRqRL algorithm, an online vector quantization (VQ) method is employed to quantize the network size to a codebook size, resulting in the quantized KRqRL (QKRqRL) algorithm. This paper provides a detailed analysis of the convergence and computational complexity of the QKRqRL algorithm. In the simulation experiments, the network size of each algorithm is reduced to 25% of its original size. The performance of the QKRqRL algorithm is evaluated in terms of convergence speed, prediction error, and computation time under non-Gaussian noise conditions. Finally, the QKRqRL algorithm is further validated using sunspot data, demonstrating its superior stability and online prediction performance.

Enhancing internet of things security using entropy-informed RF-DNA fingerprint learning from Gabor-based images

Internet of Things (IoT) deployments are anticipated to reach 29.42 billion by the end of 2030 at an average growth rate of 16% over the next 6 years. These deployments represent an overall growth of 201.4% in operational IoT devices from 2020 to 2030. This growth is alarming because IoT devices have permeated all aspects of our daily lives, and most lack adequate security. IoT-connected systems and infrastructures can be secured using device identification and authentication, two effective identity-based access control mechanisms. Physical Layer Security (PLS) is an alternative or augmentation to cryptographic and other higher-layer security schemes often used for device identification and authentication. PLS does not compromise spectral and energy efficiency or reduce throughput. Specific Emitter Identification (SEI) is a PLS scheme capable of uniquely identifying senders by passively learning emitter-specific features unintentionally imparted on the signals during their formation and transmission by the sender’s radio frequency (RF) front end. This work focuses on image-based SEI because it produces deep learning (DL) models that are less sensitive to external factors and better generalize to different operating conditions. More specifically, this work focuses on reducing the computational cost and memory requirements of image-based SEI with little to no reduction in performance by selecting the most informative portions of each image using entropy. These image portions or tiles reduce memory storage requirements by 92.8% and the DL training time by 81% while achieving an average percent correct classification performance of 91% and higher for SNR values of 15 dB and higher with individual emitter performance no lower than 87.7% at the same SNR. Compared with another state-of-the-art time-frequency (TF)-based SEI approach, our approach results in superior performance for all investigated signal-to-noise ratio conditions, the largest improvement being 21.7% at 9 dB and requires 43% less data.

Hybrid Filtering and Probabilistic Techniques for Privacy-Preserving Community Detection in OSNs

Online Social Networks (OSNs) face the major challenge of protecting participant's privacy, due to the high dimensionality and volume of the data. In real-time social networks, where hundreds of personal details of people are shared every day, there remains a significant threat to privacy. Privacy preservation is challenging for a community detection problem due to the high computational complexity and memory requirements, especially in larger real-world OSN graphs. Although weighted nodes provide better results, as they allow capturing the frequencies of the values, the privacy preservation of sensitive attributes such as specific profiles becomes harder compared to these models. This problem can result in queries and subsequent learnings from social network profiles of specific individuals, which may be of personal, political or otherwise concern. Online social networks (OSNs) grapple with significant privacy challenges due to the extensive dimensions and vast quantities of data involved. This research fills the void in current privacy-preserving community detection methodologies, which face problems in computational complexity and memory usage in large-scale OSNs. The proposed framework seeks to bolster privacy preservation through a comprehensive multi-step process. Data filtering uses a blended data filter system to remove outliers and irrelevant data, thus enhancing the quality of the input data. Density-Oriented Clustering phase employs a density-oriented clustering model to identify communities, with each cluster representing a distinct community. Privacy Preservation component introduces a new privacy preservation technique for sensitive OSN attributes, surpassing existing k-anonymization methods. The developed density-based social network community detection model and its novel privacy-preserving scheme are evaluated using the datasets Yelp, Football, Zachary and Dolphin from the SNAP dataset. Experimental results on these datasets embed a comprehensive evaluation based on the order of each node and the graph-based networks, where each node is laden with weights as proximity values, indicating the semantic proximity between communities and individuals. The proposed framework employs the normalized mutual information (NMI), modularity (Q), Rand index and runtime measurements to demonstrate widespread advantages in the multi-dimensional functional space, including greater accuracy, cluster compatibility, and computational tractability over the existing prominent traditional models for OSNs.

Powerful numerical method for analysis of electromagnetic scattering from multiple 3D coated targets buried under rough surface

ABSTRACT In this work, an efficient numerical method is proposed for three-dimensional (3D) electromagnetic scattering from multiple coated targets buried under a two-dimensional dielectric rough surface. In the proposed method, the interior region of each coated target is regarded as an independent subregion. The field within each coated target is modeled by 3D vector finite element method (FEM). The multiple scattering interactions between the buried targets and between targets and rough surface are considered by the electric field integral equation and the Poggio–Miller–Chang–Harrington–Wu integral equations, respectively. The fields inside and outside the targets are coupled together via boundary conditions on the boundary of each target. To improve the calculation efficiency and reduce the memory requirement, the fast multipole technique is used in our method. The presented method is numerically validated by comparing it with method of moments and conventional FEM-boundary integral method (BIM). The results indicate that, compared with traditional FEM-BIM, our method has significant advantages in computational efficiency and memory requirements.

Dual Domain Decomposition Method for High-Resolution 3D Simulation of Groundwater Flow and Transport

The high-resolution 3D groundwater flow and transport simulation problem requires massive discrete linear systems to be solved, leading to significant computational time and memory requirements. The domain decomposition method is a promising technique that facilitates the parallelization of problems with minimal communication overhead by dividing the computation domain into multiple subdomains. However, directly utilizing a domain decomposition scheme to solve massive linear systems becomes impractical due to the bottleneck in algebraic operations required to coordinate the results of subdomains. In this paper, we propose a two-level domain decomposition method, named dual-domain decomposition, to efficiently solve the massive discrete linear systems in high-resolution 3D groundwater simulations. The first level of domain decomposition partitions the linear system problem into independent linear sub-problems across multiple subdomains, enabling parallel solutions with significantly reduced complexity. The second level introduces a domain decomposition preconditioner to solve the linear system, known as the Schur system, used to coordinate results from subdomains across their boundaries. This additional level of decomposition parallelizes the preconditioning of the Schur system, addressing the bottleneck of the Schur system solution while improving its convergence rates. The dual-domain decomposition method facilitates the partition and distribution of the computation to be solved into independent finely grained computational subdomains, substantially reducing both computational and memory complexity. We demonstrate the scalability of our proposed method through its application to a high-resolution 3D simulation of chromium contaminant transport in groundwater. Our results indicate that our method outperforms both the vanilla domain decomposition method and the algebraic multigrid preconditioned method in terms of runtime, achieving up to 8.617× and 5.515× speedups, respectively, in solving massive problems with approximately 108 million degrees of freedom. Therefore, we recommend its effectiveness and reliability for high-resolution 3D simulations of groundwater flow and transport.

Deep Learning and Neural Architecture Search for Optimizing Binary Neural Network Image Super Resolution.

The evolution of super-resolution (SR) technology has seen significant advancements through the adoption of deep learning methods. However, the deployment of such models by resource-constrained devices necessitates models that not only perform efficiently, but also conserve computational resources. Binary neural networks (BNNs) offer a promising solution by minimizing the data precision to binary levels, thus reducing the computational complexity and memory requirements. However, for BNNs, an effective architecture is essential due to their inherent limitations in representing information. Designing such architectures traditionally requires extensive computational resources and time. With the advancement in neural architecture search (NAS), differentiable NAS has emerged as an attractive solution for efficiently crafting network structures. In this paper, we introduce a novel and efficient binary network search method tailored for image super-resolution tasks. We adapt the search space specifically for super resolution to ensure it is optimally suited for the requirements of such tasks. Furthermore, we incorporate Libra Parameter Binarization (Libra-PB) to maximize information retention during forward propagation. Our experimental results demonstrate that the network structures generated by our method require only a third of the parameters, compared to conventional methods, and yet deliver comparable performance.

Tricking AI chips into simulating the human brain: A detailed performance analysis

In recent years, significant strides in Artificial Intelligence (AI) have led to various practical applications, primarily centered around training and deployment of deep neural networks (DNNs). These applications, however, require considerable computational resources, predominantly reliant on modern Graphics-Processing Units (GPUs). Yet, the quest for larger and faster DNNs has spurred the creation of specialized AI chips and efficient Machine-Learning (ML) software tools like TensorFlow and PyTorch have been developed for striking a balance between usability and performance. Simultaneously, the field of computational neuroscience shares a similar quest for increased computational power to simulate more extensive and detailed brain models, while also keeping usability high. Although GPUs have also entered this field, programming complexity remains high, resulting in cumbersome simulations. Inspired by AI progress, we introduce a workflow for easily accelerating brain simulations using TensorFlow and evaluate the performance of various, cutting-edge AI chips – including the Graphcore Intelligence-Processing Unit (IPU), GroqChip, Nvidia GPU with Tensor Cores, and Google Tensor-Processing Unit (TPU) – when simulating a biologically detailed as well as simpler brain models. Our model simulations explore the architectural tradeoffs of a modern-day CPU and these four AI platforms by varying computational density, memory requirements and floating-point numerical accuracy. Results show that the GroqChip achieves the best performance for small networks, yet is unable to simulate large-scale networks. At the scale of mammalian brains, the GPU, IPU and TPU achieve speedups ranging from 29x to 1,208x times over CPU runtimes. Remarkably, the TPU sets a new record for the largest, real-time simulation of the inferior-olivary nucleus in the brain. Reduced-accuracy floating-point implementations make some simulation results unreliable for brain research, notably for the GroqChip. Consequently, this work underscores the potential of ML libraries for accelerating brain simulations as well as the critical role of AI-chip numerical accuracy for biophysically realistic brain models.

Efficient in Vivo Neural Signal Compression Using an Autoencoder-Based Neural Network.

Conventional in vivo neural signal processing involves extracting spiking activity within the recorded signals from an ensemble of neurons and transmitting only spike counts over an adequate interval. However, for brain-computer interface (BCI) applications utilizing continuous local field potentials (LFPs) for cognitive decoding, the volume of neural data to be transmitted to a computer imposes relatively high data rate requirements. This is particularly true for BCIs employing high-density intracortical recordings with hundreds or thousands of electrodes. This article introduces the first autoencoder-based compression digital circuit for the efficient transmission of LFP neural signals. Various algorithmic and architectural-level optimizations are implemented to significantly reduce the computational complexity and memory requirements of the designed in vivo compression circuit. This circuit employs an autoencoder-based neural network, providing a robust signal reconstruction. The application-specific integrated circuit (ASIC) of the in vivo compression logic occupies the smallest silicon area and consumes the lowest power among the reported state-of-the-art compression ASICs. Additionally, it offers a higher compression rate and a superior signal-to-noise and distortion ratio.