Floating Point Operations Research Articles

Nested attention network based on category contexts learning for semantic segmentation

The attention mechanism is widely used in the field of semantic segmentation, due to the fact that it can be used to obtain effective long-distance dependencies by assigning different weights to objects according to different tasks. We propose a novel Nested Attention Network (NANet) for semantic segmentation, which combines Feature Category Attention (FCA) and Channel Relationship Attention (CRA) to effectively aggregate same-category contexts in both spatial and channel dimensions. Specifically, FCA captures the dependencies between spatial pixel features and categories to achieve the aggregation of features of the same category. CRA further captures the channel relationships on the output of FCA to obtain richer contexts. Numerous experiments have shown that NANet has a lower number of parameters and computational complexity than other state-of-the-art methods, and is a lightweight model with a lower total number of floating-point operations. We evaluated the performance of NANet on three datasets: Cityscapes, PASCAL VOC 2012, and ADE20K, and the experimental results show that NANet obtains promising results, reaching a performance of 82.6% on the Cityscapes test set.

A Lightweight Pine Wilt Disease Detection Method Based on Vision Transformer-Enhanced YOLO

Pine wilt disease (PWD) is a forest disease characterized by rapid spread and extremely high lethality, posing a serious threat to the ecological security of China’s forests and causing significant economic losses in forestry. Given the extensive forestry area, limited personnel for inspection and monitoring, and high costs, utilizing UAV-based remote sensing monitoring for diseased trees represents an effective approach for controlling the spread of PWD. However, due to the small target size and uneven scale of pine wilt disease, as well as the limitations of real-time detection by drones, traditional disease tree detection algorithms based on RGB remote sensing images do not achieve an optimal balance among accuracy, detection speed, and model complexity due to real-time detection limitations. Consequently, this paper proposes Light-ViTeYOLO, a lightweight pine wilt disease detection method based on Vision Transformer-enhanced YOLO (You Only Look Once). A novel lightweight multi-scale attention module is introduced to construct an EfficientViT feature extraction network for global receptive field and multi-scale learning. A novel neck network, CACSNet(Content-Aware Cross-Scale bidirectional fusion neck network), is designed to enhance the detection of diseased trees at single granularity, and the loss function is optimized to improve localization accuracy. The algorithm effectively reduces the number of parameters and giga floating-point operations per second (GFLOPs) of the detection model while enhancing overall detection performance. Experimental results demonstrate that compared with other baseline algorithms, Light-ViTeYOLO proposed in this paper has the least parameter and computational complexity among related algorithms, with 3.89 MFLOPs and 7.4 GFLOPs, respectively. The FPS rate is 57.9 (frames/s), which is better than the original YOLOv5. Meanwhile, its mAP@0.5:0.95 is the best among the baseline algorithms, and the recall and mAP@0.5 slightly decrease. Our Light-ViTeYOLO is the first lightweight method specifically designed for detecting pine wilt disease. It not only meets the requirements for real-time detection of pine wilt disease outbreaks but also provides strong technical support for automated forestry work.

Investigation on a lightweight defect detection model for photovoltaic panel

The detection of defect types of photovoltaic (PV) panel is a crucial task in PV system. Existing detection models face challenges in effectively balancing the trade-off between detection accuracy and resource consumption. To address this issue, this paper proposes a new defect detection method for PV panel based on the improved YOLOv8 model, which realizes both the high detection accuracy and the lightweight. Firstly, Reversible Column Networks (RevCol) is used as the Backbone of YOLOv8, which makes sure to preserve the feature information in the process of network transmission and also reduces the number of parameters and Giga floating-point operations per second (GFLOPs). Subsequently, a new lightweight Bottleneck fused with Efficient Multi-Scale Attention (EMA) is designed to optimize the CSPDarknet53 to 2-Stage FPN (C2f) module of Neck in YOLOv8 to enhance the robustness and further decrease network parameters. Finally, Squeeze-and-Excitation (SE) Attention is integrated into the Head of YOLOv8 to prioritize the important channel features and thus enhance the detection performance. The experimental results on the PVEL-AD dataset demonstrate that parameters and GFLOPs of the proposed model are declined by 38.46% and 34.39% respectively, and mAP0.5:0.95 is increased by 2.6% compared with the baseline model. The lightweight improved YOLOv8 model facilitates the deployment of deep learning model on edge devices and provides a novel approach for the online detection of PV panel defects.

Weight-adaptive joint mixed-precision quantization and pruning for neural network-based equalization in short-reach direct detection links.

Neural network (NN)-based equalizers have been widely applied for dealing with nonlinear impairments in intensity-modulated direct detection (IM/DD) systems due to their excellent performance. However, the computational complexity (CC) is a major concern that limits the real-time application of NN-based receivers. In this Letter, we propose, to our knowledge, a novel weight-adaptive joint mixed-precision quantization and pruning approach to reduce the CC of NN-based equalizers, where only integer arithmetic is taken into account instead of floating-point operations. The NN connections are either directly cutoff or represented by a proper number of quantization bits by weight partitioning, leading to a hybrid compressed sparse network that computes much faster and consumes less hardware resources. The proposed approach is verified in a 50-Gb/s 25-km pulse amplitude modulation (PAM)-4 IM/DD link using a directly modulated laser (DML) in the C-band. Compared with the traditional fully connected NN-based equalizer operated with standard floating-point arithmetic, about 80% memory can be saved at a minimum network size without degrading the system performance. Quantization is also shown to be more suitable to over-parameterized NN-based equalizers compared with NNs selected at a minimum size.

Green AI‐Driven Concept for the Development of Cost‐Effective and Energy‐Efficient Deep Learning Method: Application in the Detection of Eimeria Parasites as a Case Study

Although large‐scale pretrained convolutinal neural networks (CNN) models have shown impressive transfer learning capabilities, they come with drawbacks such as high energy consumption and computational cost due to their potential redundant parameters. This study presents an innovative weight‐level pruning technique that mitigates the challenges of overparameterization, and subsequently minimizes the electricity usage of such large deep learning models. The method focuses on removing redundant parameters while upholding model accuracy. This methodology is applied to classify Eimeria species parasites from fowls and rabbits. By leveraging a set of 27 pretrained CNN models with a number of parameters between 3.0M and 118.5M, the framework has identified a 4.8M‐parameter model with the highest accuracy for both animals. The model is then subjected to a systematic pruning process, resulting in an 8% reduction in parameters and a 421M reduction in floating point operations while maintaining the same classification accuracy for both fowls and rabbits. Furthermore, unlike the existing literature where two separate models are created for rabbits and fowls, this article presents a combined model with 17 classes. This approach has resulted in a CNN model with nearly 50% reduced parameter size while retaining the same accuracy of over 90%.

LCADNet: a novel light CNN architecture for EEG-based Alzheimer disease detection.

Alzheimer's disease (AD) is a progressive and incurable neurologi-cal disorder with a rising mortality rate, worsened by error-prone, time-intensive, and expensive clinical diagnosis methods. Automatic AD detection methods using hand-crafted Electroencephalogram (EEG) signal features lack accuracy and reliability. A lightweight convolution neural network for AD detection (LCADNet) is investigated to extract disease-specific features while reducing the detection time. The LCADNet uses two convolutional layers for extracting complex EEG features, two fully connected layers for selecting disease-specific features, and a softmax layer for predicting AD detection probability. A max-pooling layer interlaced between convolutional layers decreases the time-domain redundancy in the EEG signal. The efficiency of the LCADNet and four pre-trained models using transfer learning is compared using a publicly available AD detection dataset. The LCADNet shows the lowest computation complexity in terms of both the number of floating point operations and inference time and the highest classification performance across six measures. The generalization of the LCADNet is assessed by cross-testing it with two other publicly available AD detection datasets. It outperforms existing EEG-based AD detection methods with an accuracy of 98.50%. The LCADNet may be a valuable aid for neurologists and its Python implemen- tation can be found at github.com/SandeepSangle12/LCADNet.git.

Toward efficient and lightweight sea–land segmentation for remote sensing images

Sea–land segmentation is of great significance for autonomous coastline monitoring, which is fundamental research in the remote sensing community. Due to the diverse contents and easily confused sea–land boundaries contained in remote sensing images, it is always challenging to achieve precise sea–land segmentation for complex scenarios. Although existing deep learning-based methods have exhibited promising performance, excessive computational load and insufficient use of hierarchical features remain unresolved. In this paper, we contribute to addressing the problems by developing an efficient and lightweight convolutional neural network (CNN) termed E-Net. On the one hand, the proposed network adopts a novel E-shaped architecture that reforms the conventional U-codec structure to make full use of hierarchical features at different depths, so that the sea–land segmentation effect can be significantly improved without excessive computational load. On the other hand, a contextual aggregation attention mechanism (CA2M) is designed to further facilitate efficient aggregation and transmission of contextual information, so that the fuzzy and irregular sea–land boundaries can be accurately distinguished. Extensive experiments reveal that our approach not only produces superior sea–land segmentation effect but also demonstrates promising computational efficiency. Specifically, the proposed E-Net achieves state-of-the-art sea–land segmentation performance with 92.78% and 93.62% mean Intersection over Union (mIoU) on the SLSD and HRSC2016 datasets, respectively, while the frames per second (FPS) reaches 108.032 with as low as 52.287G floating point operations per second (FLOPs).

Lightweight network for millimeter-level concrete crack detection with dense feature connection and dual attention

Development of lightweight deep learning crack detection method is essential for the future deployment of mobile device-based structure inspection. The primary challenge involves the analysis and extraction of features from narrow cracks, typically 3–6 pixels wide, which are often obscured by noise such as water stains and shadows. The lightweight model should also maintain high accuracy while ensuring low computational complexity and a minimal number of parameters. To this end, this paper proposes YOLO v5-DE (Dense Feature Enhancement Connection, Efficient and Fast Convolution), a lightweight network based on the YOLO v5 architecture tailored to address these challenges, and constructs crack datasets captured at different heights to investigate the impact of different shooting distances on network performance. The network utilizes efficient convolutions and dense feature connections, with strategic reuse of filtered features from shallow layers, to significantly enhance the model's fine-grained feature information and gradient flow. The experimental results demonstrate that YOLO v5-DE achieves a detection accuracy of 96% for cracks in concrete structures. Compared to the improved YOLO v5 with EfficientViT as the backbone network, YOLO v5-DE achieves 4.7% increase on accuracy while requiring fewer computational resources, with only 1.4 million parameters and 3.6 Giga Floating point Operations Per Second (GFLOPS). Additionally, YOLO v5-DE reduces the inference time to 3.38 ms and increases the frame rate to 295.8 FPS. Moreover, the proposed lightweight network exhibits better detection performance when facing complex backgrounds and real-world environments.

LRMSNet: A New Lightweight Detection Algorithm for Multi-Scale SAR Objects

In recent years, deep learning has found widespread application in SAR image object detection. However, when detecting multi-scale targets against complex backgrounds, these models often struggle to strike a balance between accuracy and speed. Furthermore, there is a continuous need to enhance the performance of current models. Hence, this paper proposes LRMSNet, a new multi-scale target detection model designed specifically for SAR images in complex backgrounds. Firstly, the paper introduces an attention module designed to enhance contextual information aggregation and capture global features, which is integrated into a backbone network with an expanded receptive field for improving SAR image feature extraction. Secondly, this paper develops an information aggregation module to effectively fuse different feature layers of the backbone network. Lastly, to better integrate feature information at various levels, this paper designs a multi-scale aggregation network. We validate the effectiveness of our method on three different SAR object detection datasets (MSAR-1.0, SSDD, and HRSID). Experimental results demonstrate that LRMSNet achieves outstanding performance with a mean average accuracy (mAP) of 95.2%, 98.9%, and 93.3% on the MSAR-1.0, SSDD, and HRSID datasets, respectively, with only 3.46 M parameters and 12.6 G floating-point operation cost (FLOPs). When compared with existing SAR object detection models on the MSAR-1.0 dataset, LRMSNet achieves state-of-the-art (SOTA) performance, showcasing its superiority in addressing SAR detection challenges in large-scale complex environments and across various object scales.

Structured ramp secret sharing schemata over rings of real polynomials

Two new ramp secret sharing schemata based on polynomials are proposed. For both schemata, the secret is considered to be a polynomial created by the dealer. The participants are separated into ℓ⩾2, groups, that are specified by the dealer's levels Li for i=1,2,…,ℓ and each level Li, i⩾2, is separated into subsets. The shares of the secret are given to participants in the form of polynomials. For the first proposed scheme, the dealer creates ℓ polynomials one for each level. Specific participants from every subset of each level have to cooperate all together in order to construct the polynomial of their level. Next all the authorized participants cooperate for computing the greatest common divisor of the polynomials in order to retrieve the secret. In the second scheme, the authorized participants cooperate per two levels using a bottom-up procedure. In both schemata the greatest common divisor can be evaluated by implementing numerical linear algebra methods, and precisely factorization of matrices of special form such as Sylvester matrices. The triangularization of these matrices can be obtained by exploiting their special structure for the reduction of the required floating point operations. The innovative idea of the paper at hand is the use of real polynomials in secret sharing schemata. This is particularly useful since the greatest common divisor can always be computed with efficient accuracy using effective numerical methods. New theoretical results are proved and provided that support the error analysis of our approach.

A Novel Attention‐Based Layer Pruning Approach for Low‐Complexity Convolutional Neural Networks

Deep learning (DL) has been very successful for classifying images, detecting targets, and segmenting regions in high‐resolution images such as whole slide histopathology images. However, analysis of such high‐resolution images requires very high DL complexity. Several AI optimization techniques have been recently proposed that aim at reducing the complexity of deep neural networks and hence expedite their execution and eventually allow the use of low‐power, low‐cost computing devices with limited computation and memory resources. These methods include parameter pruning and sharing, quantization, knowledge distillation, low‐rank approximation, and resource efficient architectures. Rather than pruning network structures including filters, layers, and blocks of layers based on a manual selection of a significance metric such as l1‐norm and l2‐norm of the filter kernels, novel highly efficient AI‐driven DL optimization algorithms using variations of the squeeze and excitation in order to prune filters and layers of deep models such as VGG‐16 as well as eliminate filters and blocks of residual networks such as ResNet‐56 are introduced. The proposed techniques achieve significantly higher reduction in the number of learning parameters, the number of floating point operations, and memory space as compared to the‐state‐of‐the‐art methods.

L1RR: Model Pruning Using Dynamic and Self-Adaptive Sparsity for Remote-Sensing Target Detection to Prevent Target Feature Loss

Limited resources for edge computing platforms in airborne and spaceborne imaging payloads prevent using complex image processing models. Model pruning can eliminate redundant parameters and reduce the computational load, enhancing processing efficiency on edge computing platforms. Current challenges in model pruning for remote-sensing object detection include the risk of losing target features, particularly during sparse training and pruning, and difficulties in maintaining channel correspondence for residual structures, often resulting in retaining redundant features that compromise the balance between model size and accuracy. To address these challenges, we propose the L1 reweighted regularization (L1RR) pruning method. Leveraging dynamic and self-adaptive sparse modules, we optimize L1 sparsity regularization, preserving the model’s target feature information using a feature attention loss mechanism to determine appropriate pruning ratios. Additionally, we propose a residual reconstruction procedure, which removes redundant feature channels from residual structures while maintaining the residual inference structure through output channel recombination and input channel recombination, achieving a balance between model size and accuracy. Validation on two remote-sensing datasets demonstrates significant reductions in parameters and floating point operations (FLOPs) of 77.54% and 65%, respectively, and a 48.5% increase in the inference speed on the Jetson TX2 platform. This framework optimally maintains target features and effectively distinguishes feature channel importance compared to other methods, significantly enhancing feature channel robustness for difficult targets and expanding pruning applicability to less difficult targets.

Towards efficient and robust face recognition through attention-integrated multi-level CNN

AbstractThe rapid advancement of deep Convolutional Neural Networks (CNNs) has led to remarkable progress in computer vision, contributing to the development of numerous face verification architectures. However, the inherent complexity of these architectures, often characterized by millions of parameters and substantial computational demands, presents significant challenges for deployment on resource-constrained devices. To address these challenges, we introduce RobFaceNet, a robust and efficient CNN designed explicitly for face recognition (FR). The proposed RobFaceNet optimizes accuracy while preserving computational efficiency, a balance achieved by incorporating multiple features and attention mechanisms. These features include both low-level and high-level attributes extracted from input face images and aggregated from multiple levels. Additionally, the model incorporates a newly developed bottleneck that integrates both channel and spatial attention mechanisms. The combination of multiple features and attention mechanisms enables the network to capture more significant facial features from the images, thereby enhancing its robustness and the quality of facial feature extraction. Experimental results across state-of-the-art FR datasets demonstrate that our RobFaceNet achieves higher recognition performance. For instance, RobFaceNet achieves 95.95% and 92.23% on the CA-LFW and CP-LFW datasets, respectively, compared to 95.45% and 92.08% for very deep ArcFace model. Meanwhile, RobFaceNet exhibits a more lightweight model complexity. In terms of computation cost, RobFaceNet has 337M Floating Point Operations Per Second (FLOPs) compared to ArcFace’s 24211M, with only 3% of the parameters. Consequently, RobFaceNet is well-suited for deployment across various platforms, including robots, embedded systems, and mobile devices.

A lightweight Yunnan Xiaomila detection and pose estimation based on improved YOLOv8.

Yunnan Xiaomila is a pepper variety whose flowers and fruits become mature at the same time and multiple times a year. The distinction between the fruits and the background is low and the background is complex. The targets are small and difficult to identify. This paper aims at the problem of target detection of Yunnan Xiaomila under complex background environment, in order to reduce the impact caused by the small color gradient changes between xiaomila and background and the unclear feature information, an improved PAE-YOLO model is proposed, which combines the EMA attention mechanism and DCNv3 deformable convolution is integrated into the YOLOv8 model, which improves the model's feature extraction capability and inference speed for Xiaomila in complex environments, and achieves a lightweight model. First, the EMA attention mechanism is combined with the C2f module in the YOLOv8 network. The C2f module can well extract local features from the input image, and the EMA attention mechanism can control the global relationship. The two complement each other, thereby enhancing the model's expression ability; Meanwhile, in the backbone network and head network, the DCNv3 convolution module is introduced, which can adaptively adjust the sampling position according to the input feature map, contributing to stronger feature capture capabilities for targets of different scales and a lightweight network. It also uses a depth camera to estimate the posture of Xiaomila, while analyzing and optimizing different occlusion situations. The effectiveness of the proposed method was verified through ablation experiments, model comparison experiments and attitude estimation experiments. The experimental results indicated that the model obtained an average mean accuracy (mAP) of 88.8%, which was 1.3% higher than that of the original model. Its F1 score reached 83.2, and the GFLOPs and model sizes were 7.6G and 5.7MB respectively. The F1 score ranked the best among several networks, with the model weight and gigabit floating-point operations per second (GFLOPs) being the smallest, which are 6.2% and 8.1% lower than the original model. The loss value was the lowest during training, and the convergence speed was the fastest. Meanwhile, the attitude estimation results of 102 targets showed that the orientation was correctly estimated exceed 85% of the cases, and the average error angle was 15.91°. In the occlusion condition, 86.3% of the attitude estimation error angles were less than 40°, and the average error angle was 23.19°. The results show that the improved detection model can accurately identify Xiaomila targets fruits, has higher model accuracy, less computational complexity, and can better estimate the target posture.

PHSI-RTDETR: A Lightweight Infrared Small Target Detection Algorithm Based on UAV Aerial Photography

To address the issues of low model accuracy caused by complex ground environments and uneven target scales and high computational complexity in unmanned aerial vehicle (UAV) aerial infrared image target detection, this study proposes a lightweight UAV aerial infrared small target detection algorithm called PHSI-RTDETR. Initially, an improved backbone feature extraction network is designed using the lightweight RPConv-Block module proposed in this paper, which effectively captures small target features, significantly reducing the model complexity and computational burden while improving accuracy. Subsequently, the HiLo attention mechanism is combined with an intra-scale feature interaction module to form an AIFI-HiLo module, which is integrated into a hybrid encoder to enhance the focus of the model on dense targets, reducing the rates of missed and false detections. Moreover, the slimneck-SSFF architecture is introduced as the cross-scale feature fusion architecture of the model, utilizing GSConv and VoVGSCSP modules to enhance adaptability to infrared targets of various scales, producing more semantic information while reducing network computations. Finally, the original GIoU loss is replaced with the Inner-GIoU loss, which uses a scaling factor to control auxiliary bounding boxes to speed up convergence and improve detection accuracy for small targets. The experimental results show that, compared to RT-DETR, PHSI-RTDETR reduces model parameters by 30.55% and floating-point operations by 17.10%. Moreover, detection precision and speed are increased by 3.81% and 13.39%, respectively, and mAP50, impressively, reaches 82.58%, demonstrating the great potential of this model for drone infrared small target detection.

Toward digital design at the exascale: An overview of project ICECap

High performance computing has entered the Exascale Age. Capable of performing over 1018 floating point operations per second, exascale computers, such as El Capitan, the National Nuclear Security Administration's first, have the potential to revolutionize the detailed in-depth study of highly complex science and engineering systems. However, in addition to these kind of whole machine “hero” simulations, exascale systems could also enable new paradigms in digital design by making petascale hero runs routine. Currently, untenable problems in complex system design, optimization, model exploration, and scientific discovery could all become possible. Motivated by the challenge of uncovering the next generation of robust high-yield inertial confinement fusion (ICF) designs, project ICECap (Inertial Confinement on El Capitan) attempts to integrate multiple advances in machine learning (ML), scientific workflows, high performance computing, GPU-acceleration, and numerical optimization to prototype such a future. Built on a general framework, ICECap is exploring how these technologies could broadly accelerate scientific discovery on El Capitan. In addition to our requirements, system-level design, and challenges, we describe some of the key technologies in ICECap, including ML replacements for multiphysics packages, tools for human-machine teaming, and algorithms for multifidelity design optimization under uncertainty. As a test of our prototype pre-El Capitan system, we advance the state-of-the art for ICF hohlraum design by demonstrating the optimization of a 17-parameter National Ignition Facility experiment and show that our ML-assisted workflow makes design choices that are consistent with physics intuition, but in an automated, efficient, and mathematically rigorous fashion.

Reversible-Prior-Based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction

The task of reconstructing a 3D cube from a 2D measurement is not well-defined in spectral imaging. Unfortunately, existing Deep Unfolding Network (DU) and End-to-End (E2E) approaches can't strike an optimal balance between computational complexity and reconstruction quality. The goal of this study is to think about ways to merge the E2E's violent mapping with DU's iterative method. Our proposed deep learning framework, the Reversible-prior-based Spectral-Spatial Transformer, combines the high-quality reconstruction capabilities of DU with the advantages of having fewer parameters and lower computing cost, similar to the E2E approach. SST-ReversibleNet uses a reversible prior to project the end-to-end mapping reconstruction results back into the measurement space, construct the residuals between the reprojection and the actual measurement, and improve reconstruction accuracy. Extensive trials show that our SST-ReversibleNet outperforms cutting-edge approaches by at least 0.8 dB and only use 34.3% Params and 44.1% giga floating-point operations per second (GFLOP).

Fast hardware-aware matrix-free algorithms for higher-order finite-element discretized matrix multivector products on distributed systems

Recent hardware-aware matrix-free algorithms for higher-order finite-element (FE) discretized matrix-vector multiplications reduce floating point operations and data access costs compared to traditional sparse matrix approaches. In this work, we address a critical gap in existing matrix-free implementations which are not well suited for the action of FE discretized matrices on very large number of vectors. In particular, we propose efficient matrix-free algorithms for evaluating FE discretized matrix-multivector products on both multi-node CPU and GPU architectures. To this end, we employ batched evaluation strategies, with the batchsize tailored to underlying hardware architectures, leading to better data locality and enabling further parallelization. On CPUs, we utilize even-odd decomposition, SIMD vectorization, and overlapping computation and communication strategies. On GPUs, we develop strategies to overlap compute with data movement for achieving efficient pipelining and reduced data accesses through the use of GPU-shared memory, constant memory and kernel fusion. Our implementation outperforms the baselines for Helmholtz operator action on 1024 vectors, achieving up to 1.4x improvement on one CPU node and up to 2.8x on one GPU node, while reaching up to 4.4x and 1.5x improvement on multiple nodes for CPUs (3072 cores) and GPUs (24 GPUs), respectively. We further benchmark the performance of the proposed implementation for solving a model eigenvalue problem for 1024 smallest eigenvalue-eigenvector pairs by employing the Chebyshev Filtered Subspace Iteration method, achieving up to 1.5x improvement on one CPU node and up to 2.2x on one GPU node while reaching up to 3.0x and 1.4x improvement on multi-node CPUs (3072 cores) and GPUs (24 GPUs), respectively.

DyFish-DETR: Underwater Fish Image Recognition Based on Detection Transformer

Due to the complexity of underwater environments and the lack of training samples, the application of target detection algorithms to the underwater environment has yet to provide satisfactory results. It is crucial to design specialized underwater target recognition algorithms for different underwater tasks. In order to achieve this goal, we created a dataset of freshwater fish captured from multiple angles and lighting conditions, aiming to improve underwater target detection of freshwater fish in natural environments. We propose a method suitable for underwater target detection, called DyFish-DETR (Dynamic Fish Detection with Transformers). In DyFish-DETR, we propose a DyFishNet (Dynamic Fish Net) to better extract fish body texture features. A Slim Hybrid Encoder is designed to fuse fish body feature information. The results of ablation experiments show that DyFishNet can effectively improve the mean Average Precision (mAP) of model detection. The Slim Hybrid Encoder can effectively improve Frame Per Second (FPS). Both DyFishNet and the Slim Hybrid Encoder can reduce model parameters and Floating Point Operations (FLOPs). In our proposed freshwater fish dataset, DyFish-DETR achieved a mAP of 96.6%. The benchmarking experimental results show that the Average Precision (AP) and Average Recall (AR) of DyFish-DETR are higher than several state-of-the-art methods. Additionally, DyFish-DETR, respectively, achieved 99%, 98.8%, and 83.2% mAP in other underwater datasets.

PIPE: Parallelized inference through ensembling of residual quantization expansions

Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point operations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only support specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose PIPE, a quantization method that leverages residual error expansion, along with group sparsity and an ensemble approximation for better parallelization. PIPEis backed off by strong theoretical guarantees and achieves superior performance on every benchmarked application (from vision to NLP tasks), architecture (ConvNets, transformers) and bit-width (from int8 to ternary quantization).