Lightweight Inference Research Articles

13‐1: A Lightweight Inference Network‐based Algorithm for Low‐Light Image Brightness Adjustment

13‐1: A Lightweight Inference Network‐based Algorithm for Low‐Light Image Brightness Adjustment

Adaptive Offloading of Transformer Inference for Weak Edge Devices with Masked Autoencoders

Transformer is a popular machine learning model used by many intelligent applications in smart cities. However, it has high computational complexity and it would be hard to deploy it in weak-edge devices. This paper presents a novel two-round offloading scheme, called A-MOT, for efficient transformer inference. A-MOT only samples a small part of image data and sends it to edge servers, with negligible computational overhead at edge devices. The image is recovered by the server with the masked autoencoder (MAE) before the inference. In addition, an SLO-adaptive module is intended to achieve personalized transmission and effective bandwidth utilization. To avoid the large overhead on the repeat inference in the second round, A-MOT further contains a lightweight inference module to save inference time in the second round. Extensive experiments have been conducted to verify the effectiveness of the A-MOT.

FPGA based Flexible Implementation of Light Weight Inference on Deep Convolutional Neural Networks

Standard Convolution (StdConv) is the main technique used in the state of the art Deep Convolutional Neural Networks (DCNNs). Fewer computations are achieved if Depthwise Separable Convolution technique (SepConv) is used as an alternative. A crucial issue in many applications like smart cameras and autonomous vehicles where low latency is essential stems from deploying a lightweight and low cost inference models. An acceptable accuracy should be kept with tolerable computations and memory access load. A flexible architecture for different DCNN convolution types and models is proposed. The flexibility comes from the sharing of one memory access unit with different types of layers regardless of the selected kernel size, by multiplying each weight vector by local operators with variant aperture. Moreover, one depthwise computation unit can be used for both standard and pointwise layers. The learnable parameters are quantized to 8-bits fixed point representation and that gives very limited reduction of accuracy and a considerable reduction of the Field-Programmable Gate Array (FPGA) resources. To reduce processing time, inter layer parallel computations are performed. The experiment is conducted by using grey scale ORL database with shallow Convolutional Neural Network (CNN) and the colored Canadian Institute for Advanced Research 10 classes (CIFAR-10) database with DCNN, and a comparable accuracies of 93% and 85.7% are achieved respectively using very low cost of Spartan 3E and moderate cost of zynq FPGA platforms

LEAN: Real-Time Analysis of Resistance Training Using Wearable Computing.

The use of fitness apps to track physical exercise has been proven to promote weight loss and increase physical activity. The most popular forms of exercise are cardiovascular training and resistance training. The overwhelming majority of cardio tracking apps automatically track and analyse outdoor activity with relative ease. In contrast, nearly all commercially available resistance tracking apps only record trivial data, such as the exercise weight and repetition number via manual user input, a level of functionality not far from that of a pen and paper. This paper presents LEAN, a resistance training app and exercise analysis (EA) system for both the iPhone and Apple Watch. The app provides form analysis using machine learning, automatic repetition counting in real time, and other important but seldom studied exercise metrics, such as range of motion on a per-repetition level and average repetition time. All features are implemented using lightweight inference methods that enable real-time feedback on resource-constrained devices. The performance evaluation includes a user survey and benchmarking of all data science features using both ground-truth data from complementary modalities and comparisons with commercial apps.

SBNN: A Searched Binary Neural Network for SAR Ship Classification

The synthetic aperture radar (SAR) for ocean surveillance missions requires low latency and light weight inference. This paper proposes a novel small-size Searched Binary Network (SBNN), with network architecture search (NAS) for ship classification with SAR. In SBNN, convolution operations are modified by binarization technologies. Both input feature maps and weights are quantized into 1-bit in most of the convolution computation, which significantly decreases the overall computational complexity. In addition, we propose a patch shift processing, which can adjust feature maps with learnable parameters at spatial level. This process enhances the performance by reducing the information irrelevant to the targets. Experimental results on the OpenSARShip dataset show the proposed SBNN outperforms both binary neural networks from computer vision and CNN-based SAR ship classification methods. In particular, SBNN shows a great advantage in computational complexity.

Lightweight inference for automation efficiency

We explore the role of lightweight inference techniques in creating highly automated engineering support environments for the development of computer-based systems. Lightweight inference techniques are scalable methods for automated reasoning. We outline the types of automation that would be enabled by effective lightweight inference capabilities and survey some promising approaches to realizing the needed capabilities.

Preface: Volume 25

Preface: Volume 25