Lightweight Computation Research Articles

A Scaled Monocular 3D Reconstruction Based on Structure from Motion and Multi-View Stereo

Three-dimensional digital modeling at actual scales is essential for digitally preserving cultural relics. While 3D reconstruction using a monocular camera offers a cost-effective solution, the lack of scale information in the resulting models limits their suitability for geometric measurements. Objects with monotonous textures, such as batteries, pose additional challenges due to insufficient feature points, increasing positional uncertainty. This article proposes a method incorporating point and line features to address the scale ambiguity in multi-view 3D reconstruction using monocular cameras. By pre-measuring the lengths of multiple sets of real line segments, building a lookup table, and associating the line features in different images, the table was input into the improved reconstruction algorithm to further optimize the scale information. Experimental results on real datasets showed that the proposed method outperformed the COLMAP method by 70.82% in reconstruction accuracy, with a scale recovery reaching millimeter-level accuracy. This method is highly generalizable, cost-effective, and supports lightweight computation, making it suitable for real-time operation on a CPU.

A Global Spatial-Spectral Feature Fused Autoencoder for Nonlinear Hyperspectral Unmixing

Hyperspectral unmixing (HU) aims to decompose mixed pixels into a set of endmembers and corresponding abundances. Deep learning-based HU methods are currently a hot research topic, but most existing unmixing methods still rely on per-pixel training or employ convolutional neural networks (CNNs), which overlook the non-local correlations of materials and spectral characteristics. Furthermore, current research mainly focuses on linear mixing models, which limits the feature extraction capability of deep encoders and further improvement in unmixing accuracy. In this paper, we propose a nonlinear unmixing network capable of extracting global spatial-spectral features. The network is designed based on an autoencoder architecture, where a dual-stream CNNs is employed in the encoder to separately extract spectral and local spatial information. The extracted features are then fused together to form a more complete representation of the input data. Subsequently, a linear projection-based multi-head self-attention mechanism is applied to capture global contextual information, allowing for comprehensive spatial information extraction while maintaining lightweight computation. To achieve better reconstruction performance, a model-free nonlinear mixing approach is adopted to enhance the model’s universality, with the mixing model learned entirely from the data. Additionally, an initialization method based on endmember bundles is utilized to reduce interference from outliers and noise. Comparative results on real datasets against several state-of-the-art unmixing methods demonstrate the superior of the proposed approach.

Deep Smart Cities: A Review of Smart Cities Applications-Based an Inference of Deep Learning Upon IoT Devices

Recent years have witnessed an increase in the use of IoT devices in Smart Cities (SC). With the remarkable success of integrating deep learning (DL) in various fields of SC such as smart environment, smart agriculture, and more. There is an emerging field to deploy DL into IoT devices to meet the requirement of SC applications as real-time applications and lightweight computation. However, the limited resource of IoT devices poses a challenge to fulfilling the DL models that demand large storage and massive computation. The aim of this paper is focused on the integration of DL with IoT in SC and conducts up-to-date state-of-the-art studies on the SC applications. First, a present overview of SC, IoT and DL. Second, we review studies that integrate DL and IoT into four SC applications (smart environment, smart transportation, smart agriculture, and smart home). It covers a series of crucial applications in SC such as Air pollution, water pollution, autonomous driving, activity recognition and detection, indoor localization, and more. Third, present the main challenges once integrate DL on IoT devices in SC. Lastly, we introduced open issues for the future direction obtained when integrate DL models with IoT devices in SC applications to inspire further research in this area.

Enhancing privacy and security in smart healthcare: A blockchain-powered decentralized data dissemination scheme

Dissemination of health data in smart healthcare allows a patient’s health information to be accessible across healthcare communities for the improvement of patients’ care services. In our work, data collected from a patient’s automated health device is distributed to the communities through the blockchain network. Blockchain technology paves the way for the health sector to run in a decentralized system. To achieve a secure decentralized data dissemination, the transmitted health data must be reliable, that is, data is assured originated from a legitimate device without unlawful alteration. However, verification of such reliability may jeopardize a patient’s privacy. In healthcare, privacy is of the utmost importance to protect sensitive health data. At the same time, should misbehavior arise, malicious entities should be held accountable. We satisfy these conflicting requirements of reliability, privacy and accountability with our proposed secure decentralized data dissemination scheme for smart healthcare that integrates a certificate-based signcryption with a proxy re-encryption (Sign-Proxy) scheme and an Improved Delegated Proof-of-Stake (DPoS) consensus protocol for the system construction. The Sign-Proxy provides users a desirable level of security while delivering lightweight computation for limited computing capabilities of automated health devices. In parallel, the Improved DPoS facilitates the streamlined and secure adoption of a consortium blockchain framework. The security and performance analysis indicate that the proposed scheme is resilience against adversarial attacks, achieves overall efficiency, and feasible for real-world deployment.

Co-design based FPGA implementation of an efficient new speech hyperchaotic cryptosystem in the transform domain

In this paper a new encryption system has been designed and implemented for real-time speech transmission to reduce bandwidth requirements, increase security and minimize residual intelligibility. To guarantee robustness and lightweight computation, the developed cryptosystem has been carried out in the wavelet transform domain based on a hyperchaotic model to generate mask and permutation keys. The cryptographic system has been designed using a hardware-software (HW/SW) co-design approach by developing several IP-cores in a relatively short development time. The performances and security evaluation of the system have been validated through simulation results followed by an experimental validation through the implementation of an encrypted speech signal transmission between two low cost Nexys-4 DDR FPGA platforms, operating in real-time for both wired and wireless communications. Compared to similar works, high performances have been obtained in terms of bandwidth efficiency due to the use of DWT, limited area of FPGA resources, low power consumption and high security level with a large keyspace that is sufficient to resist against brute force attacks. The designed system can be a very useful solution for many real-time secure integrated voice communication systems, multiple communication purposes, military, professional or personal high level of conversations security.

ScopeViT: Scale-Aware Vision Transformer

Multi-scale features are essential for various vision tasks, such as classification, detection, and segmentation. Although Vision Transformers (ViTs) show remarkable success in capturing global features within an image, how to leverage multi-scale features in Transformers is not well explored. This paper proposes a scale-aware vision Transformer called ScopeViT that efficiently captures multi-granularity representations. Two novel attention with lightweight computation are introduced: Multi-Scale Self-Attention (MSSA) and Global-Scale Dilated Attention (GSDA). MSSA embeds visual tokens with different receptive fields into distinct attention heads, allowing the model to perceive various scales across the network. GSDA enhances model understanding of the global context through token-dilation operation, which reduces the number of tokens involved in attention computations. This dual attention method enables ScopeViT to “see” various scales throughout the entire network and effectively learn inter-object relationships, reducing heavy quadratic computational complexity. Extensive experiments demonstrate that ScopeViT achieves competitive complexity/accuracy trade-offs compared to existing networks across a wide range of visual tasks. On the ImageNet-1K dataset, ScopeViT achieves a top-1 accuracy of 81.1%, using only 7.4M parameters and 2.0G FLOPs. Our approach outperforms Swin (ViT-based) by 1.9% accuracy while saving 42% of the parameters, outperforms MobileViTv2 (Hybrid-based) with a 0.7% accuracy gain while using 50% of the computations, and also beats ConvNeXt V2 (ConvNet-based) by 0.8% with fewer parameters.

Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models

Although recent personalization methods have democratized high-resolution image synthesis by enabling swift concept acquisition with minimal examples and lightweight computation, they also present an exploitable avenue for highly accessible backdoor attacks. This paper investigates a critical and unexplored aspect of text-to-image (T2I) diffusion models - their potential vulnerability to backdoor attacks via personalization. By studying the prompt processing of popular personalization methods (epitomized by Textual Inversion and DreamBooth), we have devised dedicated personalization-based backdoor attacks according to the different ways of dealing with unseen tokens and divide them into two families: nouveau-token and legacy-token backdoor attacks. In comparison to conventional backdoor attacks involving the fine-tuning of the entire text-to-image diffusion model, our proposed personalization-based backdoor attack method can facilitate more tailored, efficient, and few-shot attacks. Through comprehensive empirical study, we endorse the utilization of the nouveau-token backdoor attack due to its impressive effectiveness, stealthiness, and integrity, markedly outperforming the legacy-token backdoor attack.

SDN-enabled Quantized LQR for Smart Traffic Light Controller to Optimize Congestion

Existing intersection management systems, in urban cities, lack in meeting the current requirements of self-configuration, lightweight computing, and software-defined control, which are necessarily required for congested road-lane networks. To satisfy these requirements, this work proposes effective, scalable, multi-input and multi-output, and congestion prevention-enabled intersection management system utilizing a software-defined control interface that not only regularly monitors the traffic to prevent congestion for minimizing queue length and waiting time but also offers a computationally efficient solution in real-time. For effective intersection management, a modified linear-quadratic regulator, i.e., Quantized Linear Quadratic Regulator (QLQR), is designed along with Software-defined Networking (SDN)-enabled control interface to maximize throughput and vehicles speed and minimize queue length and waiting time at the intersection. Experimental results prove that the proposed SDN-QLQR improves the comparative performance in the interval of 24.94%–49.07%, 35.78%–68.86%, 36.67%–59.08%, and 29.94%–57.87% for various performance metrics, i.e., average queue length, average waiting time, throughput, and average speed, respectively.

Key point calibrating and clustering for hard example mining of dense analogs in the fish-eye lens.

Recent years have witnessed widespread applications of the fish-eye lens with a wide field-of-view. However, its inherent distortion poses a big challenge to the intelligent recognition of dense analogs (IRDA) by convolutional neural networks (CNN). The major bottleneck of existing CNN models lies in their limited modeling capacity for distorted objects in fish-eye images, leading to the misclassification of hard examples. To further improve the accuracy of IRDA, we propose a novel key point calibrating and clustering (KPCC) algorithm based on the hemispherical projection model. Our method can effectively correct the hard example misclassification predicted by the CNN, significantly enhancing the performance of the IRDA. The experiments show that, as a light-weight computation calibrating and stable adaptive clustering method, the KPCC increases the precision and recall rate of IRDA on the intelligent retail dataset by 8.55% and 8.07%, respectively; compared with the classic Focalloss, QFocalloss, and OHEM (online hard example mining), it can mine hard examples more sufficiently, especially in the scene of distorted dense analog detection.

Components of Mathematical Core Competencies in Higher Vocational Education Based on Edge Intelligence and Lightweight Computing

Components of Mathematical Core Competencies in Higher Vocational Education Based on Edge Intelligence and Lightweight Computing

PartialHD: Toward Efficient Hyperdimensional Computing by Partial Processing

Hyperdimensional (HD) computing is a brain inspired learning method that is widely employed in resource-constrained applications such as the internet of things (IoT) regarding its lightweight computation. Although HD computing has high efficiency in the IoT applications, it suffers from high computation due to the large vector size. Thus, several studies are proposed to speedup and increase the efficiency of HD computing. In this work, we propose a method to process HD computing called PartialHD. Our method divides a long hypervector into multiple partial vectors, and process each vector separately. In the retraining phase, this method improves accuracy up to 1.93%. Moreover, by employing our proposed method in the retraining and inference, only a few partial vectors participate and this causes the computational overhead reduces in these phases. The evaluation shows that our method processes on average 22.2% and 36.23% of the entire hypervectors with negligible accuracy loss in inference and retraining, respectively. Furthermore, we propose two general architectures (light-weight and high-speed), which accelerate partial vector computations on different FPGA platforms. The result shows that the light-weight architecture can accelerate HD computing on resource-constrained FPGAs such as Artix while the high-speed architecture has 4.72× higher throughput than light-weight architecture.

A hybrid TCN-GRU model for classifying human activities using smartphone inertial signals.

Recognising human activities using smart devices has led to countless inventions in various domains like healthcare, security, sports, etc. Sensor-based human activity recognition (HAR), especially smartphone-based HAR, has become popular among the research community due to lightweight computation and user privacy protection. Deep learning models are the most preferred solutions in developing smartphone-based HAR as they can automatically capture salient and distinctive features from input signals and classify them into respective activity classes. However, in most cases, the architecture of these models needs to be deep and complex for better classification performance. Furthermore, training these models requires extensive computational resources. Hence, this research proposes a hybrid lightweight model that integrates an enhanced Temporal Convolutional Network (TCN) with Gated Recurrent Unit (GRU) layers for salient spatiotemporal feature extraction without tedious manual feature extraction. Essentially, dilations are incorporated into each convolutional kernel in the TCN-GRU model to extend the kernel's field of view without imposing additional model parameters. Moreover, fewer short filters are applied for each convolutional layer to alleviate excess parameters. Despite reducing computational cost, the proposed model utilises dilations, residual connections, and GRU layers for longer-term time dependency modelling by retaining longer implicit features of the input inertial sequences throughout training to provide sufficient information for future prediction. The performance of the TCN-GRU model is verified on two benchmark smartphone-based HAR databases, i.e., UCI HAR and UniMiB SHAR. The model attains promising accuracy in recognising human activities with 97.25% on UCI HAR and 93.51% on UniMiB SHAR. Since the current study exclusively works on the inertial signals captured by smartphones, future studies will explore the generalisation of the proposed TCN-GRU across diverse datasets, including various sensor types, to ensure its adaptability across different applications.

PRLAP-IoD: A PUF-based Robust and Lightweight Authentication Protocol for Internet of Drones

In recent years, the industry and research have cast great attention to the Internet of Drones (IoD), which is becoming progressively popular since it can bring a great convenience to various application scenarios, such as national map exploration, public safety monitoring and automated military applications. In these scenarios, Unmanned Aerial Vehicles (UAVs) (called drones) will be used to collect private information. Due to the fact that the private information are very sensitive, and drones working in public places easily suffer from physical capture or tampering attacks, the primary concern is that this information could be collected by adversaries or unauthorized users. In addition, as resource-constrained devices, drones are mostly equipped with small memory and have limited computing power. Therefore, how to ensure robust security as much as possible while achieving lightweight computing and communication costs has become an urgent problem to be solved in this field. In this paper, we propose A PUF-based Robust and Lightweight Authentication Protocol for Drone-Gateway and Drone-Drone Communication (PRLAP-IoD) to address these issues. Both formal security validation using conventional tools (ROR Model and AVISPA) and other informal security analysis clearly demonstrate that PRLAP-IoD can not only provide physical security, but also defend against a variety of known attacks. Finally, compared with the recent Authentication and Key Agreement (AKA) schemes, PRLAP-IoD can attain a delicate balance between computation cost and communication cost in IoD environment.

Reinforcement learning based CPG-controlled method with high adaptability and robustness: An experimental study on a robotic fishtail

An adaptive and robust control is imperative for robots operating in complex environments. The artificial intelligence approach presents a promising solution, while training and practical application in an actual robot remain a significant challenge. This paper proposes a reinforcement learning based control strategy with lightweight computation, aiming to optimize thrust in complex flows and varying self-structural characteristics of robotic fishtails. Improved Q-learning algorithm and CPG control are utilized to enable the robotic fishtail to make autonomous and accurate decisions in response to unknown changes. The control strategy is trained in actual physical flow fields, with an action selection strategy and a reward system proposed to accelerate convergence and enhance training stability. An integrated system of online learning, response measuring, and real-time monitoring is developed for the training and testing processes. Variable environmental tests are performed under different turbulent flows. Distinct caudal fins are utilized to conduct tests on self-structural variation. The experimental results confirm the robustness and adaptability of the control strategy, as well as its superiority over the PID approach in terms of accuracy, stability, and response speed. The control strategy architecture and physical experimental method could offer experience and application reference for the intelligent control of actual robots.

Room-aware portable Auditory Augmented Reality: Real-time spatial audio generation with geometric data analysis

This study proposes an innovative and practical approach to spatial acoustic rendering for Auditory Augmented Reality (AAR) in small room usage. AAR aims to enhance user experiences by overlaying spatially appropriate sounds in the real environment, creating the illusion that virtual sounds coexist in the physical world. However, current technological constraints in hardware and AAR rendering present challenges in creating a truly immersive sensation of “being there.” To address these challenges, we present a dynamic processing framework for a typical mobile AAR setting. This framework combines a game-audio engine with 3-D visual computation both specialized for real-time processing, providing a plausible spatial impression with lightweight computation. The process involves capturing room information by the Light Detection And Ranging (LiDAR) sensor mounted on the device, followed by estimating its dominant acoustic materials. The generated data will be used to compute dynamic early reflections and late reverberation. Early reflections are calculated up to the fourth reflection using the image source method, considering the acoustic materials. Late reverberation is generated using the Feedback Delay Network (FDN), with room frequency characteristics and spatial directivity varying relative to the listener's position. All components will be represented in fifth-order Ambisonics and binauralized with head-tracking. Notably, this dynamic rendering method could be light enough to handle all processing on a single mobile device.

MEGADOCK-on-Colab: an easy-to-use protein–protein docking tool on Google Colaboratory

MotivationSince the advent of ColabFold, numerous software packages have been provided with Google Colaboratory-compatible ipynb files, allowing users to effortlessly test and reproduce results without the need for local installation or configuration. MEGADOCK, a protein–protein docking tool, is particularly well-suited for Google Colaboratory due to its lightweight computations and GPU acceleration capabilities. To increase accessibility and promote widespread use, it is crucial to provide a computing environment compatible with Google Colaboratory.ResultsIn this study, we report the development of a Google Colaboratory environment for running our protein–protein docking software, MEGADOCK. We provide a comprehensive ipynb file, including the compilation of MEGADOCK with the FFTW library installation on Colaboratory, the introduction of related tools using PyPI/apt, and the execution and visualization of docking structures. This streamlined environment enables users to visualize docking structures with just one click. The code is available under a CC-BY NC 4.0 license from https://github.com/ohuelab/MEGADOCK-on-Colab.

Privacy-Preserving Parallel Computation of Matrix Determinant With Edge Computing

With the widespread deployment of secure outsourcing computation, the resource-constrained client can delegate intensive computation tasks to powerful servers. Matrix determinant computation is a fundamental mathematical operation that has been widely used in IoT applications. This operation is computationally expensive. Nevertheless, the existing secure outsourcing protocols for matrix determinant are all designed based on one cloud server, which cannot well meet the low-latency and real-time computing requirements for the client. To address this issue, we explore accelerating the computation of matrix determinant by parallel outsourcing based on two nearby edge servers and propose the first practical protocol. We use the matrix blocking technique to split the computation task into multiple subtasks, which are parallel outsourced to edge servers for accelerating the computation. Moreover, we propose a privacy-preserving matrix transformation technique for data privacy protection. This technique only involves the operations of matrix-vector multiplication and matrix-matrix addition. It achieves the lightweight computation for the client and supports computational indistinguishability for the blinded input and a uniform distribution. The correctness, privacy and verifiability of the proposed protocol are analyzed. Finally, the performance advantage of the proposed protocol is demonstrated through simulation experiments.

CIRLEM: a synergic integration of Collective Intelligence and Reinforcement learning in Energy Management for enhanced climate resilience and lightweight computation

A novel energy management (EM) approach is introduced, integrating core elements of collective intelligence (CI) and reinforcement learning (RL) and called CIRLEM. It operates by distributing a flexibility signal from the energy supplier to agents within the grid, prompting their responsive actions. The flexibility signal reflects upon the collective behaviour of the agents in the grid and agents learn and decide using a value-based model-free RL engine. Two ways of running CIRLEM are defined, based on doing all the decision making only at the edge node (Edge Node Control or ENC) or together with the cluster (Edge node and Cluster Control or ECC). CIRLEM's performance is thoroughly investigated in an elderly building situated in Ålesund, Norway, specifically during extreme warm and cold seasons in the future climate. The building is divided into 20 thermal zones, each acting as an agent with three control strategies. CIRLEM undergoes comprehensive testing, evaluating policies with 24 and 48 sets of actions (referred to as L24 and L48) and six different randomness levels. The results demonstrate that CIRLEM swiftly converges to an optimal solution (the optimum set of policies), offering both enhanced indoor comfort and significant energy savings. Among the CIRLEM algorithms, ENC-L24, the fastest and simplest one, showcased outstanding performance. Overall, CIRLEM offers a remarkable improvement in energy flexibility and climate resilience for a group of grid-connected agents, ensuring energy savings without compromising indoor comfort.

A Human Activity Recognition Method Based on Lightweight Feature Extraction Combined With Pruned and Quantized CNN for Wearable Device

Human Activity Recognition (HAR) is becoming an essential part of human life care. Existing HAR methods are usually developed using a two-level approach, wherein a first-level Machine Learning (ML) classifier is employed to distinguish the static and dynamic activities, followed by a second-level classifier to identify the specific activity. These approaches are not suitable for wearable devices, due to the high computational and memory consumption. Our rigorous analysis of various HAR datasets opens up a new possibility that static or dynamic activities can be discriminated against through a simple statistical technique. Therefore, we propose to utilize a statistical feature extraction technique to replace the first-level ML classifier, thus achieving more lightweight computation. Next, we employ Random Forest (RF) and Convolutional Neural Networks (CNN) to classify the specific activities, achieving higher accuracy compared to the state-of-the-art results. We further reduce the computation and memory consumption of the above combined approach by applying pruning and quantizing techniques to CNN (PQ-CNN). Experimental results show the proposed lightweight HAR method achieved an F1 score of 0.9417 and 0.9438 for unbalanced and balanced datasets, respectively. On top of lightweight and accuracy, the proposed HAR method is practical for wearable devices by using a single accelerometer.

Representation granularity enables time-efficient autonomous exploration in large, complex worlds.

We propose a dual-resolution scheme to achieve time-efficient autonomous exploration with one or many robots. The scheme maintains a high-resolution local map of the robot's immediate vicinity and a low-resolution global map of the remaining areas of the environment. We believe that the strength of our approach lies in this low- and high-resolution representation of the environment: The high-resolution local map ensures that the robots observe the entire region in detail, and because the local map is bounded, so is the computation burden to process it. The low-resolution global map directs the robot to explore the broad space and only requires lightweight computation and low bandwidth to communicate among the robots. This paper shows the strength of this approach for both single-robot and multirobot exploration. For multirobot exploration, we also introduce a "pursuit" strategy for sharing information among robots with limited communication. This strategy directs the robots to opportunistically approach each other. We found that the scheme could produce exploration paths with a bounded difference in length compared with the theoretical shortest paths. Empirically, for single-robot exploration, our method produced 80% higher time efficiency with 50% lower computational runtimes than state-of-the-art methods in more than 300 simulation and real-world experiments. For multirobot exploration, our pursuit strategy demonstrated higher exploration time efficiency than conventional strategies in more than 3400 simulation runs with up to 20 robots. Last, we discuss how our method was deployed in the DARPA Subterranean Challenge and demonstrated the fastest and most complete exploration among all teams.