Limited Computational Resources Research Articles

Research on the Construction of an Efficient and Lightweight Online Detection Method for Tiny Surface Defects through Model Compression and Knowledge Distillation

In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression and knowledge distillation is proposed. Firstly, data augmentation is employed in the preprocessing stage to increase the diversity of training samples, thereby improving the model’s robustness and generalization capability. The K-means++ clustering algorithm generates candidate bounding boxes, adapting to defects of different sizes and selecting finer features earlier. Secondly, the cross stage partial (CSP) Darknet53 network and spatial pyramid pooling (SPP) module extract features from the input raw images, enhancing the accuracy of defect location detection and recognition in YOLO. Finally, the concept of model compression is integrated, utilizing scaling factors in the batch normalization (BN) layer, and introducing sparse factors to perform sparse training on the network. Channel pruning and layer pruning are applied to the sparse model, and post-processing methods using knowledge distillation are used to effectively reduce the model size and forward inference time while maintaining model accuracy. The improved model size decreases from 244 M to 4.19 M, the detection speed increases from 32.8 f/s to 68 f/s, and mAP reaches 97.41. Experimental results demonstrate that this method is conducive to deploying network models on embedded devices with limited GPU computing and storage resources. It can be applied in distributed service architectures for edge computing, providing new technological references for deploying deep learning models in the industrial sector.

YOLOv8-CB: Dense Pedestrian Detection Algorithm Based on In-Vehicle Camera

Recently, the field of vehicle-mounted visual intelligence technology has witnessed a surge of interest in pedestrian detection. Existing algorithms for dense pedestrian detection at intersections face challenges such as high computational weight, complex models that are difficult to deploy, and suboptimal detection accuracy for small targets and highly occluded pedestrians. To address these issues, this paper proposes an improved lightweight multi-scale pedestrian detection algorithm, YOLOv8-CB. The algorithm introduces a lightweight cascade fusion network, CFNet (cascade fusion network), and a CBAM attention module to improve the characterization of multi-scale feature semantics and location information, and it superimposes a bidirectional weighted feature fusion path BIFPN structure to fuse more effective features and improve pedestrian detection performance. It is experimentally verified that compared with the YOLOv8n algorithm, the accuracy of the improved model is increased by 2.4%, the number of model parameters is reduced by 6.45%, and the computational load is reduced by 6.74%. The inference time for a single image is 10.8 ms. The cascade fusion algorithm YOLOv8-CB has higher detection accuracy and is a lighter model for multi-scale pedestrian detection in complex scenes such as streets or intersections. This proposed algorithm presents a valuable approach for device-side pedestrian detection with limited computational resources.

Multiple Moving Vehicles Tracking Algorithm with Attention Mechanism and Motion Model

With the acceleration of urbanization and the increasing demand for travel, current road traffic is experiencing rapid growth and more complex spatio-temporal logic. Vehicle tracking on roads presents several challenges, including complex scenes with frequent foreground–background transitions, fast and nonlinear vehicle movements, and the presence of numerous unavoidable low-score detection boxes. In this paper, we propose AM-Vehicle-Track, following the proven-effective paradigm of tracking by detection (TBD). At the detection stage, we introduce the lightweight channel block attention mechanism (LCBAM), facilitating the detector to concentrate more on foreground features with limited computational resources. At the tracking stage, we innovatively propose the noise-adaptive extended Kalman filter (NSA-EKF) module to extract vehicles’ motion information while considering the impact of detection confidence on observation noise when dealing with nonlinear motion. Additionally, we borrow the Byte data association method to address unavoidable low-score detection boxes, enabling secondary association to reduce ID switches. We achieve 42.2 MOTA, 51.2 IDF1, and 364 IDs on the test set of VisDrone-MOT with 72 FPS. The experimental results showcase our approach’s highly competitive performance, attaining SOTA tracking performance with a fast speed.

IPR-VINS: Real-time monocular visual-inertial SLAM with implicit plane optimization

SLAM typically involves updating numerous landmarks to reconstruct scenes rather than generating a topological depiction of the actual environment. Planar features, prevalent in structured environments, can contribute to the development of more nuanced yet less cumbersome maps. The incorporation of fusions between feature points and planar facets gives rise to new discrepancies that must be meticulously addressed. To prevent the uncontrolled expansion of planar boundaries, a robust technique for boundary extraction is offered. Scene planes are fused using the boundary function, ensuring stable feature extraction and planar association. To leverage limited computational resources effectively, a streamlined back-end framework is constructed. Finally, factors are fine-tuned at a consistent scale, which not only elevates locational accuracy but also diminishes computational complexity. To reduce systematic errors, plane prior information is assimilated into global optimization. Compared with VINS-Mono, the average Absolute Trajectory Error of the proposed method on EuRoC is reduced by 44%.

An ultra-fast and energy-efficient CNTFET-based image corner detection hardware for real-time image processing applications

In image processing and machine vision, corner detection is pivotal in diverse applications, including computer vision, 3D reconstruction, face detection, object tracking, and video technologies. Despite the wide usage, the real-time and energy-efficient hardware implementation of corner detection algorithms remains a critical challenge because of the computational resource limitations. On the other hand, owing to the complicated nature of corner detection algorithms, their hardware implementation has been limited to the graphics processing unit (GPU) and field programmable gate arrays (FPGA) platforms. In this regard, this work aims to propose a novel and ultra-efficient carbon nanotube field-effect transistor (CNTFET)-based hardware for image corner detection. Thanks to the proposed corner detection algorithm, the designed hardware has been realized using 2742 transistors with competitive accuracy. The proposed corner detection hardware indicated a remarkable salt-and-pepper noise immunity without using any noise reduction circuit. Our comprehensive simulations demonstrate 78%, 87%, and 94.5% total average improvements in delay, power, and energy compared to the other related corner detection hardware. Moreover, the proposed CNTFET-based corner detection hardware shows a 43 ps propagation delay, demonstrating its real-time operation. The proposed corner detection algorithm at the system level shows suitable accuracy metrics such as Recall, Precision, and error of detection (EoD) compared to the other well-known corner detectors. Our method has established a new pathway for real-time circuit-level hardware design for image processing and machine vision applications.

Application of molecular dynamics-based pharmacophore and machine learning approaches to identify novel Mcl1 inhibitors through drug repurposing and mechanics research.

Myeloid cell leukemia 1 (Mcl1), a critical protein that regulates apoptosis, has been considered as a promising target for antitumor drugs. The conventional pharmacophore screening approach has limitations in conformation sampling and data mining. Here, we offered an innovative solution to identify Mcl1 inhibitors with molecular dynamics-refined pharmacophore and machine learning methods. Considering the safety and druggability of FDA-approved drugs, virtual screening of the database was performed to discover Mcl1 inhibitors, and the hit was subsequently validated via TR-FRET, cytotoxicity, and flow cytometry assays. To reveal the binding characteristics shared by the hit and a typical Mcl1 selective inhibitor, we employed quantum mechanics and molecular mechanics (QM/MM) calculations, umbrella sampling, and metadynamics in this work. The combined studies suggested that fluvastatin had promising cell inhibitory potency and was suitable for further investigation. We believe that this research will shed light on the discovery of novel Mcl1 inhibitors that can be used as a supplemental treatment against leukemia and provide a possible method to improve the accuracy of drug repurposing with limited computational resources while balancing the costs of experimentation well.

Sketched reference databases for genome-based taxonomy and comparative genomics.

The analysis of curated genomic, metagenomic and proteomic data is of paramount importance in the fields of biology, medicine, education, and bioinformatics. Although this type of data is usually hosted in raw format on free international repositories, the full access requires lots of computing power and large storage disk space for the domestic user. The purpose of the study is to offer a comprehensive set of microbial genomic and proteomic reference databases in an accessible and easy-to-use form to the scientific community and demonstrate its advantages and usefulness. Also, we present a case study on the applicability of the sketched data, for the determination of overall genomic coherence between two members of the Brucellacea family, which suggests they belong to the same genomospecies that remain as discrete ecotypes. A representative set of genomes, proteomes (from type material), and metagenomes were directly collected from the NCBI Assembly database and Genome Taxonomy Database (GTDB), associated with the major groups of Bacteria, Archaea, Virus, and Fungi. Sketched databases were subsequently created and stored on handy reduced representations by using the MinHash algorithm implemented in Mash software. The obtained dataset contains more than 133 GB of space disk reduced to 883.25 MB and represents 125,110 genomics/proteomic records from eight informative contexts, which have been prefiltered to make them accessible, usable, and user-friendly with limited computational resources. Potential uses of these sketched databases are discussed, including but not limited to microbial species delimitation, estimation of genomic distances and genomic novelties, paired comparisons between proteomes, genomes, and metagenomes; phylogenetic neighbor's exploration and selection, among others.

Fire and smoke real-time detection algorithm for coal mines based on improved YOLOv8s.

Fire and smoke detection is crucial for the safe mining of coal energy, but previous fire-smoke detection models did not strike a perfect balance between complexity and accuracy, which makes it difficult to deploy efficient fire-smoke detection in coal mines with limited computational resources. Therefore, we improve the current advanced object detection model YOLOv8s based on two core ideas: (1) we reduce the model computational complexity and ensure real-time detection by applying faster convolutions to the backbone and neck parts; (2) to strengthen the model's detection accuracy, we integrate attention mechanisms into both the backbone and head components. In addition, we improve the model's generalization capacity by augmenting the data. Our method has 23.0% and 26.4% fewer parameters and FLOPs (Floating-Point Operations) than YOLOv8s, which means that we have effectively reduced the computational complexity. Our model also achieves a mAP (mean Average Precision) of 91.0%, which is 2.5% higher than the baseline model. These results show that our method can improve the detection accuracy while reducing complexity, making it more suitable for real-time fire-smoke detection in resource-constrained environments.

Enhanced Gradient for Differentiable Architecture Search.

In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve the network performance and reduce the network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a gradient-based relaxation method is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, an evolutionary multiobjective algorithm is utilized to complete the automatic design from blocks to the target network. The experimental results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of 3.18% on Canadian Institute for Advanced Research (CIFAR10) and an error rate of 19.16% on CIFAR100, both at network parameter size less than 1 M. Obviously, compared with other NAS methods, our method offers a tremendous reduction in designed network architecture parameters.

Verification Study of in Silico Computed Intracardiac Blood Flow With 4D Flow MRI.

Major challenges for clinical applications of in silico medicine are limitations in time and computational resources. Computational approaches should therefore be tailored to specific applications with relatively low complexity and must be verified and validated against clinical gold standards. This study performed computational fluid dynamics simulations of left ventricular hemodynamics of different complexity based on shape reconstruction from steady state gradient echo magnetic resonance imaging (MRI) data. Computed flow results of a rigid wall model (RWM) and a prescribed motion fluid-structure interaction (PM-FSI) model were compared against phase-contrast MRI measurements for three healthy subjects. Extracted boundary conditions from the steady state MRI sequences as well as computed metrics, such as flow rate, valve velocities, and kinetic energy show good agreement with in vivo flow measurements. Regional flow analysis reveals larger differences. Basic flow structures are well captured with RWM and PM-FSI. For the computation of further biomarkers like washout or flow efficiency, usage of PM-FSI is required. Regarding boundary-near flow, more accurate anatomical models are inevitable. These results delineate areas of application of both methods and lay a foundation for larger validation studies and sensitivity analysis for healthy and diseased cases, being an essential step upon clinical translations.

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices

Federated learning has been recently proposed for many clients to collaboratively train a machine learning model in a privacy-preserving manner. However, it also amplifies the difficulty in designing good neural network architecture, especially considering heterogeneous mobile devices. To this end, we propose a novel neural architecture search algorithm, namely FedNAS, which can automatically generate a set of optimal models under federated settings. The main idea is to decouple the two primary steps of the NAS process, i.e, model search and model training, and separately distribute them on the cloud and devices. It also tackles the primary challenge of limited on-device computational and communication resources through its novel designs: FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process and incorporates three key optimizations: parallel candidate training on partial clients, early dropping candidates with inferior performance, and dynamic round numbers. Evaluated on typical CNN architectures and large-scale datasets, FedNAS is able to achieve comparable model accuracy as a state-of-the-art NAS algorithm that trains models with centralized data, and also reduces the client cost by up to 200_ or more compared to a straightforward design of federated NAS.

Autoencoder-Embedded Iterated Local Search for Energy-Minimized Task Schedules of Human–Cyber–Physical Systems

This work considers a task scheduling problem with deadline constraints in human-cyber-physical systems. To find its energy-efficient schedules in a short time, an autoencoder-embedded iterated local search algorithm is proposed to solve it. Iterated local search is selected as a main scheduler. In order to handle real-time requirements and high computational load involved in the problem solution, a Long Short-Term Memory-based AutoEncoder model (LSTM-AE) is constructed to capture the relevant and complementary features of the considered problem. The model is used to find low-order and high-fit sub-solutions via unsupervised end-to-end learning, and generates promising solutions in an informative low-dimensional solution space. To further reduce computational burden, a two-stage optimization framework is constructed, which includes an off-line training phase and an online optimization one. The former trains LSTM-AE by using expert knowledge and historical data. The latter designs optimal resource allocation strategies to build a high-quality initial solution. Then, LSTM-AE-assisted local search operators are proposed and used to reform the initial solution and generate better ones. Various numerical experiments are performed to compare the proposed method with several classic heuristics and some recently-developed methods. The results show its superiority over them. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —In human-cyber-physical systems, a task scheduling problem is usually solved by using heuristics due to the limited computational resources. Nevertheless, fast dispatching rules tend to perform poorly. Meta-heuristics can find a relatively high-quality schedule but are time-consuming, especially for a population-based algorithm that requires to evaluate a fitness function for many candidate solutions at each iteration. To balance computational burden and solution quality, our idea is to combine machine-learning methods with meta-heuristics. Specially, we integrate a long short-term memory-based autoencoder model into iterated local search to improve the latter’s optimization ability. The combination of meta-heuristics and machine learning techniques makes it possible to obtain a high-quality schedule for the concerned problems in a short time. Theoretic analysis and experimental results show that the proposed method well outperforms its competitive peers.

Multitask Learning for Joint Diagnosis of Multiple Mental Disorders in Resting-State fMRI.

Facing the increasing worldwide prevalence of mental disorders, the symptom-based diagnostic criteria struggle to address the urgent public health concern due to the global shortfall in well-qualified professionals. Thanks to the recent advances in neuroimaging techniques, functional magnetic resonance imaging (fMRI) has surfaced as a new solution to characterize neuropathological biomarkers for detecting functional connectivity (FC) anomalies in mental disorders. However, the existing computer-aided diagnosis models for fMRI analysis suffer from unstable performance on large datasets. To address this issue, we propose an efficient multitask learning (MTL) framework for joint diagnosis of multiple mental disorders using resting-state fMRI data. A novel multiobjective evolutionary clustering algorithm is presented to group regions of interests (ROIs) into different clusters for FC pattern analysis. On the optimal clustering solution, the multicluster multigate mixture-of-expert model is used for the final classification by capturing the highly consistent feature patterns among related diagnostic tasks. Extensive simulation experiments demonstrate that the performance of the proposed framework is superior to that of the other state-of-the-art methods. Moreover, the potential for practical application of the framework is also validated in terms of limited computational resources, real-time analysis, and insufficient training data. The proposed model can identify the remarkable interpretative biomarkers associated with specific mental disorders for clinical interpretation analysis.

Effective Streaming Low-Tubal-Rank Tensor Approximation via Frequent Directions.

Low-tubal-rank tensor approximation has been proposed to analyze large-scale and multidimensional data. However, finding such an accurate approximation is challenging in the streaming setting, due to the limited computational resources. To alleviate this issue, this article extends a popular matrix sketching technique, namely, frequent directions (FDs), for constructing an efficient and accurate low-tubal-rank tensor approximation from streaming data based on the tensor singular value decomposition (t-SVD). Specifically, the new algorithm allows the tensor data to be observed slice by slice but only needs to maintain and incrementally update a much smaller sketch, which could capture the principal information of the original tensor. The rigorous theoretical analysis shows that the approximation error of the new algorithm can be arbitrarily small when the sketch size grows linearly. Extensive experimental results on both synthetic and real multidimensional data further reveal the superiority of the proposed algorithm compared with other sketching algorithms for getting low-tubal-rank approximation, in terms of both efficiency and accuracy.

Real-Time Semantic Segmentation via Spatial-Detail Guided Context Propagation.

Nowadays, vision-based computing tasks play an important role in various real-world applications. However, many vision computing tasks, e.g., semantic segmentation, are usually computationally expensive, posing a challenge to the computing systems that are resource-constrained but require fast response speed. Therefore, it is valuable to develop accurate and real-time vision processing models that only require limited computational resources. To this end, we propose the spatial-detail guided context propagation network (SGCPNet) for achieving real-time semantic segmentation. In SGCPNet, we propose the strategy of spatial-detail guided context propagation. It uses the spatial details of shallow layers to guide the propagation of the low-resolution global contexts, in which the lost spatial information can be effectively reconstructed. In this way, the need for maintaining high-resolution features along the network is freed, therefore largely improving the model efficiency. On the other hand, due to the effective reconstruction of spatial details, the segmentation accuracy can be still preserved. In the experiments, we validate the effectiveness and efficiency of the proposed SGCPNet model. On the Cityscapes dataset, for example, our SGCPNet achieves 69.5% mIoU segmentation accuracy, while its speed reaches 178.5 FPS on 768 x 1536 images on a GeForce GTX 1080 Ti GPU card. In addition, SGCPNet is very lightweight and only contains 0.61 M parameters. The code will be released at https://github.com/zhouyuan888888/SGCPNet.

Line-Based 3D Building Abstraction and Polygonal Surface Reconstruction From Images.

Textureless objects, repetitive patterns and limited computational resources pose significant challenges to man-made structure reconstruction from images, because feature-points-based reconstruction methods usually fail due to the lack of distinct texture or ambiguous point matches. Meanwhile multi-view stereo approaches also suffer from high computational complexity. In this paper, we present a new framework to reconstruct 3D surfaces for buildings from multi-view images by leveraging another fundamental geometric primitive: line segments. To this end, we first propose a new multi-resolution line segment detector to extract 2D line segments from each image. Then, we construct a 3D line cloud by introducing an improved Line3D++ algorithm to match 2D line segments from different images. Finally, we reconstruct a complete and manifold surface mesh from 3D line segments by formulating a Bayesian probabilistic modeling problem, which accurately generates a set of underlying planes. This output model is simple and has low performance requirements for hardware devices. Experimental results demonstrate the validity of the proposed approach and its ability to generate abstract and compact surface meshes from the 3D line cloud with low computational costs.

Integrating systems of systems with a federation of rule engines

Systems of Systems (SoSs) integrate many critical systems our society relies on. In designing individual systems, stakeholders use bespoke protocols, custom information models, and proprietary components with limited computational resources from various vendors. We present a reference architecture that allows multiple stakeholders to carry out a flexible integration without giving up control to a single entity in the presence of the aforementioned limitations. Our architecture relies on rule engines and graph data model to integrated systems flexibly even when black-box components are used. At the same time, a federation of the rule engines allows each stakeholder to retain control over the rules that reflect their policies. We also rely on a common information model based on ontologies to account for the information model mismatch and reduce the duplication of integration efforts. Moving rule execution to the standalone rule engines allows deployment in resource-constrained and proprietary environments. A uniform application programming interface (API) is used to integrate rule engines across systems as well as components within each system with a respective rule engine. We also present a novel algorithm to determine dependencies across rules deployed in different rule engines within the federation. This allows domain experts to develop rules as usual without having to deal with the distributed aspect of the system. We also present a proof of the sufficient condition to ensure all necessary notifications will be sent to ensure correct rule activation across different rule engines. Compared to other systems involving distributed rules, the proposed architecture is well-suited for the integration of transactional workloads commonly found in enterprises. The qualitative evaluation based on the Architecture Tradeoff Analysis Method (ATAM), applied to a telecommunications use case, shows that the architecture possesses the “interoperability”, “modifiability”, and the “functional completeness” quality attributes with a trade-off around rule expressiveness. The quantitative evaluation demonstrates speedup over the single-node setup in most scenarios except in case of highly optimized rules and a poor network performance simultaneously (tr<10ms, tn=100ms).

An investigation of stress shot peening using a 2D FEM-based simulation approach

In this study, a 2D Finite Element (FE) modeling approach is introduced for Stress Shot Peening (SSP), an enhanced version of Shot Peening (SP) widely used for fatigue life enhancement in leaf springs. The model incorporates the stochastic nature of peening particles and degraded surface properties, offering qualitative insights into the SSP process. Moreover, it reduces calculation time compared to a 3D approach, enabling the examination of multiple scenarios with limited computational resources. The approach is validated for both SP and SSP cases, providing a comprehensive overview of the SSP process, including parametrical studies on prestress level and peening angle.

Lightweight encryption with data and device integrity using NLFSR and PUF for the Internet of Medical Things

Nowadays the Internet of Medical Things (IoMT) is in top demand; they communicate with each other in an IoT network, but there are various security issues including privacy preservation, data and device integrity, and authentication. The IoMT devices have challenges like limited computational power and resources, so conventional techniques are not suitable for them. This paper presents a lightweight stream cipher to ensure privacy protection, data integrity, and device integrity using a message authentication code (MAC). The stream cipher is designed using a nonlinear feedback shift register (NLFSR). The cipher is associated with a MAC generated by concatenating a physical unclonable function (PUF) response and the hash of the plaintext to protect the device and data integrity. The use of NLFSR and PUF makes the proposed scheme robust and lightweight for resource-constrained devices like MIoT. To evaluate the quality of the cipher the NIST 800-22 statistical test suite (NIST STS) is used, and results are compared with existing stream ciphers like Rivest cipher 4 (RC4), RC4 based on chaotic mapping (RCCM), and product algebra RCCM (PARCCM). All the tests and comparisons were performed using a Heart Disease Dataset. The scheme ensures the confidentiality and integrity of sensitive medical data which are critical for protecting patient privacy and regulatory compliance. The proposed scheme can also resist cloning attacks on the IoMT devices.

Forecasting energy power consumption using federated learning in edge computing devices

Several studies in the literature propose using machine learning algorithms to forecast consumers’ energy consumption. However, such data is sensitive and has privacy constraints. On the other hand, federated learning is a technique in which the training of machine learning algorithms is performed locally, where the data is generated. In this context, this article presents a hybrid neural network architecture, named CNN-LSTM FED, trained using the public Smart* and The Building Data Genome Project 2 datasets. Additionally, an augmented Smart* dataset was generated using Generative Adversarial Networks (GANs). The performance of CNN-LSTM FED was evaluated by comparing it against the MultiLayer Perceptron (MLP), which serves as a baseline, and against a non-federated version of the CNN-LSTM FED, named CNN-LSTM. Our approach was able to generalize the model even when less than 1% of buildings participated in the modeling process, forecasting with good results the energy consumption of other buildings. Furthermore, the deployment of this architecture in an edge computing device, with limited computational resources for training, is evaluated.