Limited Hardware Resources Research Articles

A Video-based Lifting Action Recognition Using Rank-Altered Kinematic Feature Pairs

Low-back musculoskeletal disorders (MSDs) are the primary work-related injuries among manual material handling (MMH) workers, who are frequently exposed to repetitive lifting. To prevent low-back MSDs in the workplace, we present a video-based lifting action recognition method using rank-altered kinematic feature pairs, called top-scoring pairs (TSPs). We derive TSPs from a video dataset containing lifting and other activities commonly seen in MMH. These TSPs collectively classify each frame as lifting and non-lifting. The validation process involves evaluating classification performance. The proposed method minimizes computational and memory requirements while achieving performance comparable to more complex methods with greater computational demands. This makes it suitable for systems with limited hardware resources, thereby providing extensive feasibility across a variety of MMH environments to improve workplace safety.

Efficient Small Object Detection You Only Look Once: A Small Object Detection Algorithm for Aerial Images

Aerial images have distinct characteristics, such as varying target scales, complex backgrounds, severe occlusion, small targets, and dense distribution. As a result, object detection in aerial images faces challenges like difficulty in extracting small target information and poor integration of spatial and semantic data. Moreover, existing object detection algorithms have a large number of parameters, posing a challenge for deployment on drones with limited hardware resources. We propose an efficient small-object YOLO detection model (ESOD-YOLO) based on YOLOv8n for Unmanned Aerial Vehicle (UAV) object detection. Firstly, we propose that the Reparameterized Multi-scale Inverted Blocks (RepNIBMS) module is implemented to replace the C2f module of the Yolov8n backbone extraction network to enhance the information extraction capability of small objects. Secondly, a cross-level multi-scale feature fusion structure, wave feature pyramid network (WFPN), is designed to enhance the model’s capacity to integrate spatial and semantic information. Meanwhile, a small-object detection head is incorporated to augment the model’s ability to identify small objects. Finally, a tri-focal loss function is proposed to address the issue of imbalanced samples in aerial images in a straightforward and effective manner. In the VisDrone2019 test set, when the input size is uniformly 640 × 640 pixels, the parameters of ESOD-YOLO are 4.46 M, and the average mean accuracy of detection reaches 29.3%, which is 3.6% higher than the baseline method YOLOv8n. Compared with other detection methods, it also achieves higher detection accuracy with lower parameters.

Hardware Area Efficiently and Real-Time FPGA Implementation of PHMMRGB

For encryption applications on embedded systems, operating in real-time while using minimal system resources is essential. It is expected that efficient and rapid encryption of high-resolution images is to be accomplished with limited hardware resources. Therefore, in order to ensure the desired efficiency of encryption of high-resolution images, the system must possess a digital architecture capable of achieving high speeds with minimal hardware resources. This study considered the limitations of a hardware architecture for an image encryption algorithm based on the Profile Hidden Markov Model called PHMMRGB. The proposed architecture is relatively simple in comparison to its alternatives. The hardware architecture, designed with the objective of using minimal resources, has been implemented on an FPGA. Based on the results of the proposed architecture, it is believed that the implementation of the specified method on an FPGA would yield high efficiency. Experiments conducted using large-sized satellite images confirmed this expectation.

A Multilayer Nonlinear Permutation Framework and Its Demonstration in Lightweight Image Encryption

As information systems become more widespread, data security becomes increasingly important. While traditional encryption methods provide effective protection against unauthorized access, they often struggle with multimedia data like images and videos. This necessitates specialized image encryption approaches. With the rise of mobile and Internet of Things (IoT) devices, lightweight image encryption algorithms are crucial for resource-constrained environments. These algorithms have applications in various domains, including medical imaging and surveillance systems. However, the biggest challenge of lightweight algorithms is balancing strong security with limited hardware resources. This work introduces a novel nonlinear matrix permutation approach applicable to both confusion and diffusion phases in lightweight image encryption. The proposed method utilizes three different chaotic maps in harmony, namely a 2D Zaslavsky map, 1D Chebyshev map, and 1D logistic map, to generate number sequences for permutation and diffusion. Evaluation using various metrics confirms the method’s efficiency and its potential as a robust encryption framework. The proposed scheme was tested with 14 color images in the SIPI dataset. This approach achieves high performance by processing each image in just one iteration. The developed scheme offers a significant advantage over its alternatives, with an average NPCR of 99.6122, UACI of 33.4690, and information entropy of 7.9993 for 14 test images, with an average correlation value as low as 0.0006 and a vast key space of 2800. The evaluation results demonstrated that the proposed approach is a viable and effective alternative for lightweight image encryption.

Fast and accurate approximation algorithms for computing floating point square root

AbstractThe square root is one of the most used functions in many different engineering and scientific applications. We propose new methods for calculating the square root function that are based on the Newton–Raphson method with Heron iteration. A modification of Heron’s formula combined with an improved selection of the magic constants enables a significant reduction of the maximum relative error (MRE). Simple modifications to the Newton–Raphson formula and the magic number method enable implementation on platforms with limited hardware resources, such as microcontrollers and FPGAs, with variable accuracy. Implementations of new approximation algorithms in the C programming language were carefully tested and evaluated against their software and hardware counterparts on the most popular platforms, e.g., CPUs from Intel, AMD and ARM, GPU from Nvidia and IPU from Graphcore. The proposed numerical algorithms are shown to be superior in terms of computational time, the number of clock cycles, accuracy, MRE, and root mean square deviation.

Fabric Defect Detection Based on Improved Lightweight YOLOv8n

In response to the challenges posed by complex background textures and limited hardware resources in fabric defect detection, this study proposes a lightweight fabric defect detection algorithm based on an improved GSL-YOLOv8n model. Firstly, to reduce the parameter count and complexity of the YOLOv8n network, the GhostNet concept is used to construct the C2fGhost module, replacing the conventional convolution layers in the YOLOv8n structure with Ghost convolutions. Secondly, the SimAM parameter-free attention mechanism is embedded at the end of the backbone network to eliminate redundant background, enhance semantic information for small targets, and improve the network’s feature extraction capability. Lastly, a lightweight shared convolution detection head is designed, employing the scale layer to adjust features, ensuring the lightweight nature of the model while minimizing precision loss. Compared to the original YOLOv8n model, the improved GSL-YOLOv8n algorithm increases the mAP@0.5 by 0.60% to 98.29% and reduces model size, computational load, and parameter count by 66.7%, 58.0%, and 67.4%, respectively, meeting the application requirements for fabric defect detection in textile industry production.

Computer vision that can ‘see’ in the dark

Insufficient lighting environment has raised challenges for night shift workers’ safety monitoring. Thus, we have developed a computer vision-based algorithm recognizing 11 actions based on action recognition in dark (ARID) dataset. A hybrid model of integrating convolutional neural network (CNN) into YOLOv7 has been proposed. YOLOv7 is an algorithm designed for real-time object detection in image or video, for fast and accurate detection in applications such as autonomous vehicles and surveillance systems. In this work, video in dark environment has first been enhanced using CNN algorithm before feeding into YOLOv7 network for activity recognition. Adaptive gamma intensity correction (GIC) has been integrated to further improving the overall result. The proposed model has been evaluated over different enhancement modes. The proposed model is able to handle dark video frames with 74.95% Top-1 accuracy with fast processing speed of 93.99 ms/frame on a 4 GB RTX 3050 graphical processing unit (GPU) and 17.59 ms/frame on 16 GB Tesla T4 GPU. The base size of the proposed model is tiny, only 74.8 MB, but with 36.54 M of total parameters indicating that it has more capacity to learn more meaningful information with limited hardware resources.

SMALNet: Segment Anything Model Aided Lightweight Network for Infrared Image Segmentation

Infrared based visual perception is important for night vision of autonomous vehicles, unmanned aerial vehicles (UAVs), etc. Semantic segmentation based on deep learning is one of the key techniques for infrared vision-based perception systems. Currently, most of the advanced methods are based on Transformers, which can achieve favorable segmentation accuracy. However, the high complexity of Transformers prevents them from meeting the real-time requirement of inference speed in resource constrained applications. In view of this, we suggest several lightweight designs that significantly reduce existing computational complexity. In order to maintain the segmentation accuracy, we further introduce the recent vision big model — Segment Anything Model (SAM) to supply auxiliary supervisory signals while training models. Based on these designs, we propose a lightweight segmentation network termed SMALNet (Segment Anything Model Aided Lightweight Network). Compared to existing state-of-the-art method, SegFormer, it reduces 64% FLOPs while maintaining the accuracy to a large extent on two commonly-used benchmarks. The proposed SMALNet can be used in various infrared based vision perception systems with limited hardware resources.

An insight into the effect of different parameters for optimizing parallel reservoir simulation performance

Parallel simulation significantly reduces runtime in oil reservoir simulations, enabling higher-resolution analysis and a better understanding of reservoir behavior. This study comprehensively evaluates how simulation time, hardware configuration, and various physical parameters impact the efficiency, scalability, and performance metrics such as run time reduction (RTR) and number of optimal cores (NOC) in parallel simulations. This study introduced the reduced simulation time (RST) concept, an innovative approach that enhances parallel simulation efficiency, and dynamic load balance. In this approach, simulations were performed using only 5% of the total simulation time. The findings reveal that in 80% of the models, switching to RST mode has no effect on NOC but significantly reduces overall execution time in most cases. In complex models, this time reduction can be as high as 60% compared to serial mode. In the remaining 20%, the number of cores in the total simulation time (TST) mode may be higher than in the RST mode, but this difference can be disregarded due to limited hardware resources and no change in RTR.

Towards Robust Colour Texture Analysis with Limited Training Data

Texture analysis plays an important role in different domains of healthcare, agriculture, and industry, where multi-channel sensors are gaining more attention. This contribution presents an interpretable and efficient framework for texture classification and segmentation that exploits colour or channel information and does not require much data to produce accurate results. This makes such a framework well-suited for medical applications and resource-limited hardware. Our approach builds upon a distance-based generalized matrix learning vector quantization (GMLVQ) algorithm. We extend it with parametrized angle-based dissimilarity and introduce a special matrix format for multi-channel images. Classification accuracy evaluation of various model designs was performed on VisTex and ALOT data, and the segmentation application was demonstrated on an agricultural data set. Our extension of parametrized angle dissimilarity measure leads to better model generalization and robustness against varying lighting conditions than its Euclidean counterpart. The proposed matrix format for multichannel images enhances classification accuracy while reducing the number of parameters. Regarding segmentation, our method shows promising results, provided with a small class-imbalanced training data set. Proposed methodology achieves higher accuracy than prior work benchmarks and a small-scale CNN while maintaining a significantly lower parameter count. Notably, it is interpretable and accurate in scenarios where limited and unbalanced training data are available.

Improving real-time object detection in Internet-of-Things smart city traffic with YOLOv8-DSAF method

With the rise of global smart city construction, target detection technology plays a crucial role in optimizing urban functions and improving the quality of life. However, existing target detection technologies still have shortcomings in terms of accuracy, real-time performance, and adaptability. To address this challenge, this study proposes an innovative target detection model. Our model adopts the structure of YOLOv8-DSAF, comprising three key modules: depthwise separable convolution (DSConv), dual-path attention gate module (DPAG), and feature enhancement module (FEM). Firstly, DSConv technology optimizes computational complexity, enabling real-time target detection within limited hardware resources. Secondly, the DPAG module introduces a dual-channel attention mechanism, allowing the model to selectively focus on crucial areas, thereby improving detection accuracy in high-dynamic traffic scenarios. Finally, the FEM module highlights crucial features to prevent their loss, further enhancing detection accuracy. Additionally, we propose an Internet of Things smart city framework consisting of four main layers: the application domain, the Internet of Things infrastructure layer, the edge layer, and the cloud layer. The proposed algorithm utilizes the Internet of Things infrastructure layer, edge layer, and cloud layer to collect and process data in real-time, achieving faster response times. Experimental results on the KITTI V and Cityscapes datasets indicate that our model outperforms the YOLOv8 model. This suggests that in complex urban traffic scenarios, our model exhibits superior performance with higher detection accuracy and adaptability. We believe that this innovative model will significantly propel the development of smart cities and advance target detection technology.

Research and Implementation of FPGA Image Compression Parallel Algorithm Based on CCSDS

High-pixel, high-resolution spectral images are an important source of key graphical information obtained from space remote sensing, however, with the dramatically expanding amount of graphical information, the limited transmission bandwidth and hardware resources are difficult to meet the demand for massive data, which makes real-time image lossless compression and transmission technology an important solution. For this reason, this paper is based on the CCSDS image lossless compression algorithm, using FPGA for the construction of image compression system, and proposes a method strategy based on parallel acceleration algorithm to realize the real time processing of large-capacity hyperspectral images. The simulation results show that hyperspectral river and mountain range images can be compressed with greatly improved compression efficiency while maintaining real time performance and accuracy.

CFE-YOLOv8s: Improved YOLOv8s for Steel Surface Defect Detection

Due to the low detection accuracy in steel surface defect detection and the constraints of limited hardware resources, we propose an improved model for steel surface defect detection, named CBiF-FC-EFC-YOLOv8s (CFE-YOLOv8s), including CBS-BiFormer (CBiF) modules, Faster-C2f (FC) modules, and EMA-Faster-C2f (EFC) modules. Firstly, because of the potential information loss that convolutional neural networks (CNN) may encounter when dealing with miniature targets, the CBiF combines CNN with Transformer to optimize local and global features. Secondly, to address the increased computational complexity caused by the extensive use of convolutional layers, the FC uses the FasterNet block to reduce redundant computations and memory access. Lastly, the EMA is incorporated into the FC to design the EFC module and enhance feature fusion capability while ensuring the light weight of the model. CFE-YOLOv8s achieves mAP@0.5 values of 77.8% and 69.5% on the NEU-DET and GC10-DET datasets, respectively, representing enhancements of 3.1% and 2.8% over YOLOv8s, with reductions of 22% and 18% in model parameters and FLOPS. The CFE-YOLOv8s demonstrates superior overall performance and balance compared to other advanced models.

Real-time scheduling of power grid digital twin tasks in cloud via deep reinforcement learning

As energy demand continues to grow, it is crucial to integrate advanced technologies into power grids for better reliability and efficiency. Digital Twin (DT) technology plays a key role in this by using data to monitor and predict real-time operations, significantly enhancing system efficiency. However, as the power grid expands and digitization accelerates, the data generated by the grid and the DT system grows exponentially. Effectively handling this massive data is crucial for leveraging DT technology. Traditional local computing faces challenges such as limited hardware resources and slow processing speeds. A viable solution is to offload tasks to the cloud, utilizing its powerful computational capabilities to support the stable operation of the power grid. To address the need, we propose GD-DRL, a task scheduling method based on Deep Reinforcement Learning (DRL). GD-DRL considers the characteristics of computational tasks from the power grid and DT system and uses a DRL agent to schedule tasks in real-time across different computing nodes, optimizing for processing time and cost. We evaluate our method against several established real-time scheduling techniques, including Deep Q-Network (DQN). Our experimental results show that the GD-DRL method outperforms existing strategies by reducing response time, lowering costs, and increasing success rates.

LSDN Empowers the Metaverse Communication to Achieve High-Resolution Visuals with Lower Bandwidth

Deploying super-resolution models on metaverse terminal devices can enhance visual effects without increasing network bandwidth. However, deploying most current super-resolution networks on metaverse terminal devices with limited hardware resources poses a challenge due to their large volumes and high computing power consumption. In this paper, we present a lightweight separation and distillation network (LSDN) aimed at reducing the model complexity by prioritizing network structure. Specifically, we initially present the blueprint separable convolution (BSConv) to decrease model complexity and utilize the BSConv and information distillation mechanism building the channel separation distillation block (CSDB). Subsequently, we develop the enhanced spatial attention block (ESA) and Fused-MBConv (FMBConv) to explore latent information. In addition, we employ three CSDBs, an ESA, and an FMBConv to construct the residual attention unit (RAU). Finally, we concatenate several RAUs and amalgamate their hierarchical results, and transmit them to the upsampler for reconstructing the high-resolution images. We carried out comprehensive experiments on a range of datasets and found conclusive evidence that the LSDN outperforms state-of-the-art approaches, exhibiting notable enhancements in quantitative and qualitative terms.

A lightweight transformer based on feature fusion and global–local parallel stacked self-activation unit for bearing fault diagnosis

Due to the complex environment and limited hardware resources in the industrial practice diagnosis tasks, deploying deep learning-based models with large parameters is challenging. A novel lightweight bearing fault diagnosis method, global–local parallel transformer (GLP-Transformer), is proposed for balanced diagnostic performance within resource constraints. In this end-to-end framework, a multi-channel vibration feature fusion embedding block is designed, which extracts multi-position sensor signals to acquire richer original features. Furthermore, a multi-layer network structure with alternately stacked global–local parallel self-activation unit is presented for fault feature mapping processing at low costs. This unit integrates the parameter efficiency of convolutional operation and the global feature extraction expertise of transformer. Experimental verification is performed on publicly available and self-built data platforms. Compared with other methods, GLP-Transformer significantly reduces the requirements for storage and computational resources (Params: 48.28K, FLOPs: 2.74M) while possessing advanced generalization ability and robustness.

An Efficient Approach to Extract and Store Big Semantic Web Data Using Hadoop and Apache Spark GraphX

The volume of data is growing at an astonishingly high speed. Traditional techniques for storing and processing data, such as relational and centralized databases, have become inefficient and time-consuming. Linked data and the Semantic Web make internet data machine-readable. Because of the increasing volume of linked data and Semantic Web data, storing and working with them using traditional approaches is not enough, and this causes limited hardware resources. To solve this problem, storing datasets using distributed and clustered methods is essential. Hadoop can store datasets because it can use many hard disks for distributed data clustering; Apache Spark can be used for parallel data processing more efficiently than Hadoop MapReduce because Spark uses memory instead of the hard disk. Semantic Web data has been stored and processed in this paper using Apache Spark GraphX and the Hadoop Distributed File System (HDFS). Spark's in-memory processing and distributed computing enable efficient data analysis of massive datasets stored in HDFS. Spark GraphX allows graph-based semantic web data processing. The fundamental objective of this work is to provide a way for efficiently combining Semantic Web and big data technologies to utilize their combined strengths in data analysis and processing. First, the proposed approach uses the SPARQL query language to extract Semantic Web data from DBpedia datasets. DBpedia is a hugely available Semantic Web dataset built on Wikipedia. Secondly, the extracted Semantic Web data was converted to the GraphX data format; vertices and edges files were generated. The conversion process is implemented using Apache Spark GraphX. Third, both vertices and edge tables are stored in HDFS and are available for visualization and analysis operations. Furthermore, the proposed techniques improve the data storage efficiency by reducing the amount of storage space by half when converting from Semantic Web Data to a GraphX file, meaning the RDF size is around 133.8 and GraphX is 75.3. Adopting parallel data processing provided by Apache Spark in the proposed technique reduces the required data processing and analysis time. This article concludes that Apache Spark GraphX can enhance Semantic Web and Big Data technologies. We minimize data size and processing time by converting Semantic Web data to GraphX format, enabling efficient data management and seamless integration.

RVDR-YOLOv8: A Weed Target Detection Model Based on Improved YOLOv8

Currently, weed control robots that can accurately identify weeds and carry out removal work are gradually replacing traditional chemical weed control techniques. However, the computational and storage resources of the core processing equipment of weeding robots are limited. Aiming at the current problems of high computation and the high number of model parameters in weeding robots, this paper proposes a lightweight weed target detection model based on the improved YOLOv8 (You Only Look Once Version 8), called RVDR-YOLOv8 (Reversible Column Dilation-wise Residual). First, the backbone network is reconstructed based on RevCol (Reversible Column Networks). The unique reversible columnar structure of the new backbone network not only reduces the computational volume but also improves the model generalisation ability. Second, the C2fDWR module is designed using Dilation-wise Residual and integrated with the reconstructed backbone network, which improves the adaptive ability of the new backbone network RVDR and enhances the model’s recognition accuracy for occluded targets. Again, GSConv is introduced at the neck end instead of traditional convolution to reduce the complexity of computation and network structure while ensuring the model recognition accuracy. Finally, InnerMPDIoU is designed by combining MPDIoU with InnerIoU to improve the prediction accuracy of the model. The experimental results show that the computational complexity of the new model is reduced by 35.8%, the number of parameters is reduced by 35.4% and the model size is reduced by 30.2%, while the mAP50 and mAP50-95 values are improved by 1.7% and 1.1%, respectively, compared to YOLOv8. The overall performance of the new model is improved compared to models such as Faster R-CNN, SSD and RetinaNet. The new model proposed in this paper can achieve the accurate identification of weeds in farmland under the condition of limited hardware resources, which provides theoretical and technical support for the effective control of weeds in farmland.

Toward efficient resource utilization at edge nodes in federated learning

Federated learning (FL) enables edge nodes to collaboratively contribute to constructing a global model without sharing their data. This is accomplished by devices computing local, private model updates that are then aggregated by a server. However, computational resource constraints and network communication can become a severe bottleneck for larger model sizes typical for deep learning (DL) applications. Edge nodes tend to have limited hardware resources (RAM, CPU), and the network bandwidth and reliability at the edge is a concern for scaling federated fleet applications. In this paper, we propose and evaluate a FL strategy inspired by transfer learning in order to reduce resource utilization on devices, as well as the load on the server and network in each global training round. For each local model update, we randomly select layers to train, freezing the remaining part of the model. In doing so, we can reduce both server load and communication costs per round by excluding all untrained layer weights from being transferred to the server. The goal of this study is to empirically explore the potential trade-off between resource utilization on devices and global model convergence under the proposed strategy. We implement the approach using the FL framework FEDn. A number of experiments were carried out over different datasets (CIFAR-10, CASA, and IMDB), performing different tasks using different DL model architectures. Our results show that training the model partially can accelerate the training process, efficiently utilizes resources on-device, and reduce the data transmission by around 75% and 53% when we train 25%, and 50% of the model layers, respectively, without harming the resulting global model accuracy. Furthermore, our results demonstrate a negative correlation between the number of participating clients in the training process and the number of layers that need to be trained on each client’s side. As the number of clients increases, there is a decrease in the required number of layers. This observation highlights the potential of the approach, particularly in cross-device use cases.

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Anomaly detection in real-time using autoencoders implemented on edge devices is exceedingly challenging due to limited hardware, energy, and computational resources. We show that these limitations can be addressed by designing an autoencoder with low-resolution non-volatile memory-based synapses and employing an effective quantized neural network learning algorithm. We further propose nanoscale ferromagnetic racetracks with engineered notches hosting magnetic domain walls (DW) as exemplary non-volatile memory-based autoencoder synapses, where limited state (5-state) synaptic weights are manipulated by spin orbit torque (SOT) current pulses to write different magnetoresistance states. The performance of anomaly detection of the proposed autoencoder model is evaluated on the NSL-KDD dataset. Limited resolution and DW device stochasticity aware training of the autoencoder is performed, which yields comparable anomaly detection performance to the autoencoder having floating-point precision weights. While the limited number of quantized states and the inherent stochastic nature of DW synaptic weights in nanoscale devices are typically known to negatively impact the performance, our hardware-aware training algorithm is shown to leverage these imperfect device characteristics to generate an improvement in anomaly detection accuracy (90.98%) compared to accuracy obtained with floating-point synaptic weights that are extremely memory intensive. Furthermore, our DW-based approach demonstrates a remarkable reduction of at least three orders of magnitude in weight updates during training compared to the floating-point approach, implying significant reduction in operation energy for our method. This work could stimulate the development of extremely energy efficient non-volatile multi-state synapse-based processors that can perform real-time training and inference on the edge with unsupervised data.