Lightweight Attention Research Articles

Attention-Based Neural Architecture Search for Person Re-Identification.

Recent years have witnessed significant progress of person reidentification (reID) driven by expert-designed deep neural network architectures. Despite the remarkable success, such architectures often suffer from high model complexity and time-consuming pretraining process, as well as the mismatches between the image classification-driven backbones and the reID task. To address these issues, we introduce neural architecture search (NAS) into automatically designing person reID backbones, i.e., reID-NAS, which is achieved via automatically searching attention-based network architectures from scratch. Different from traditional NAS approaches that originated for image classification, we design a reID-based search space as well as a search objective to fit NAS for the reID tasks. In terms of the search space, reID-NAS includes a lightweight attention module to precisely locate arbitrary pedestrian bounding boxes, which is automatically added as attention to the reID architectures. In terms of the search objective, reID-NAS introduces a new retrieval objective to search and train reID architectures from scratch. Finally, we propose a hybrid optimization strategy to improve the search stability in reID-NAS. In our experiments, we validate the effectiveness of different parts in reID-NAS, and show that the architecture searched by reID-NAS achieves a new state of the art, with one order of magnitude fewer parameters on three-person reID datasets. As a concomitant benefit, the reliance on the pretraining process is vastly reduced by reID-NAS, which facilitates one to directly search and train a lightweight reID model from scratch.

Rapid Detection of Tools of Railway Works in the Full Time Domain

Construction tool detection is an important link in the operation and maintenance management of professional facilities in public works. Due to the large number and types of construction equipment and the complex and changeable construction environment, manual checking and inventory are still required. It is very challenging to count the variety of tools in a full-time environment automatically. To solve this problem, this paper aims to develop a full-time domain target detection system based on a deep learning network for difficult, complex railway environment image recognition. First, for the different time domain images, the image enhancement network with brightness channel decision is used to set different processing weights according to the images in different time domains to ensure the robustness of image enhancement in the entire time domain. Then, in view of the collected complex environment and the overlapping placement of the construction tools, a lightweight attention module is added on the basis of YOLOX, which makes the detection more purposeful, and the features cover more parts of the object to be recognized to improve the model. Overall detection performance. At the same time, the CIOU loss function is used to consider the distance fully, overlap rate, and penalty between the two detection frames, which is reflected in the final detection results, which can bring more stable target frame regression and further improve the recognition accuracy of the model. Experiments on the railway engineering dataset show that our RYOLO achieves a mAP of 77.26% for multiple tools and a count frame rate of 32.25FPS. Compared with YOLOX, mAP increased by 3.16%, especially the AP of woven bags with a high overlap rate increased from 0.15 to 0.57. Therefore, the target detection system proposed in this paper has better environmental adaptability and higher detection accuracy in complex railway environments, which is of great significance to the development of railway engineering intelligence.

Automatic classification of gastric lesions in gastroscopic images using a lightweight deep learning model with attention mechanism and cost-sensitive learning

Magnification endoscopy with narrow-band imaging (ME-NBI) technology is widely used in the early diagnosis of precancerous lesions and gastric cancer, which is critical to reducing the incidence of gastric cancer and improving the survival rate of gastric cancer patients. The diagnosis based on ME-NBI image is mostly in manual way in clinics. In order to improve its objectivity and efficiency, here we proposed a lightweight attention mechanism deep learning model to automatically classify ME-NBI images for artificial intelligence (AI) diagnosis of early gastric cancer, low-grade intraepithelial neoplasia, and non-neoplasm. We collected 4,098 images from 990 patients for model training and validation and evaluated the performance of our model by comparisons with that of other models, such as the benchmark model. An additional 587 images from 296 patients were collected as an independent test set to further evaluate our method’s performance. The validation set showed that the overall accuracy, recall, precision, F1 score, and the area under the curve of our method were higher than those of other methods. The independent test set showed that our method achieved state-of-the-art classification for low-grade intraepithelial neoplasia (accuracy = 93.9%, sensitivity = 92.6%). Our method displayed the advantages of lightweight and high effectiveness in classifying effectiveness, which is the potential for AI diagnosis of early gastric cancer, low-grade intraepithelial neoplasia, and non-neoplasm.

ELCD: Efficient Lunar Crater Detection Based on Attention Mechanisms and Multiscale Feature Fusion Networks from Digital Elevation Models

The detection and counting of lunar impact craters are crucial for the selection of detector landing sites and the estimation of the age of the Moon. However, traditional crater detection methods are based on machine learning and image processing technologies. These are inefficient for situations with different distributions, overlaps, and crater sizes, and most of them mainly focus on the accuracy of detection and ignore the efficiency. In this paper, we propose an efficient lunar crater detection (ELCD) algorithm based on a novel crater edge segmentation network (AFNet) to detect lunar craters from digital elevation model (DEM) data. First, in AFNet, a lightweight attention mechanism module is introduced to enhance the feature extract capabilities of networks, and a new multiscale feature fusion module is designed by fusing different multi-level feature maps to reduce the information loss of the output map. Then, considering the imbalance in the classification and the distributions of the crater data, an efficient crater edge segmentation loss function (CESL) is designed to improve the network optimization performance. Lastly, the crater positions are obtained from the network output map by the crater edge extraction (CEA) algorithm. The experiment was conducted on the PyTorch platform using two lunar crater catalogs to evaluate the ELCD. The experimental results show that ELCD has a superior detection accuracy and inference speed compared with other state-of-the-art crater detection algorithms. As with most crater detection models that use DEM data, some small craters may be considered to be noise that cannot be detected. The proposed algorithm can be used to improve the accuracy and speed of deep space probes in detecting candidate landing sites, and the discovery of new craters can increase the size of the original data set.

Deep learning for classification and localization of early gastric cancer in endoscopic images

Gastric cancer, as a malignant tumor, is one of the most common cancer-related deaths worldwide with high mortality and incidence rates. Therefore, the endoscopic detection of gastric cancer at an early stage is essential. In this paper, we propose an automatic diagnosis method of early gastric cancer (EGC) based on deep learning (DL) techniques. Specifically, with the new annotated endoscopic image dataset collected from a single-center, this paper designs several DL architectures to realize the automatic analysis of EGC images. Particularly, a guided-attention deep network was introduced, derived from ResNet-50, for the accurate score prediction of EGC and the extraction of feature information. Furthermore, we combined a lightweight attention module and multi-scale feature extractor with U-Net for estimating the pixel-level segmentation of EGC pathological regions. Experiment results on the presented dataset showed outstanding performance in the involved classification and segmentation assignments with an accuracy of 98.84% and an intersection over union (IOU) of 0.64, revealing the potential applications of DL in aiding and improving EGC diagnosis using endoscopic images.

A Lightweight Real-Time Rice Blast Disease Segmentation Method Based on DFFANet

The evaluation of rice disease severity is a quantitative indicator for precise disease control, which is of great significance for ensuring rice yield. In the past, it was usually done manually, and the judgment of rice blast severity can be subjective and time-consuming. To address the above problems, this paper proposes a real-time rice blast disease segmentation method based on a feature fusion and attention mechanism: Deep Feature Fusion and Attention Network (abbreviated to DFFANet). To realize the extraction of the shallow and deep features of rice blast disease as complete as possible, a feature extraction (DCABlock) module and a feature fusion (FFM) module are designed; then, a lightweight attention module is further designed to guide the features learning, effectively fusing the extracted features at different scales, and use the above modules to build a DFFANet lightweight network model. This model is applied to rice blast spot segmentation and compared with other existing methods in this field. The experimental results show that the method proposed in this study has better anti-interference ability, achieving 96.15% MioU, a speed of 188 FPS, and the number of parameters is only 1.4 M, which can achieve a high detection speed with a small number of model parameters, and achieves an effective balance between segmentation accuracy and speed, thereby reducing the requirements for hardware equipment and realizing low-cost embedded development. It provides technical support for real-time rapid detection of rice diseases.

Road damage detection algorithm for improved YOLOv5

Road damage detection is an important task to ensure road safety and realize the timely repair of road damage. The previous manual detection methods are low in efficiency and high in cost. To solve this problem, an improved YOLOv5 road damage detection algorithm, MN-YOLOv5, was proposed. We optimized the YOLOv5s model and chose a new backbone feature extraction network MobileNetV3 to replace the basic network of YOLOv5, which greatly reduced the number of parameters and GFLOPs of the model, and reduced the size of the model. At the same time, the coordinate attention lightweight attention module is introduced to help the network locate the target more accurately and improve the target detection accuracy. The KMeans clustering algorithm is used to filter the prior frame to make it more suitable for the dataset and to improve the detection accuracy. To improve the generalization ability of the model, a label smoothing algorithm is introduced. In addition, the structure reparameterization method is used to accelerate model reasoning. The experimental results show that the improved YOLOv5 model proposed in this paper can effectively identify pavement cracks. Compared with the original model, the mAP increased by 2.5%, the F1 score increased by 2.6%, and the model volume was smaller than that of YOLOv5. 1.62 times, the parameter was reduced by 1.66 times, and the GFLOPs were reduced by 1.69 times. This method can provide a reference for the automatic detection method of pavement cracks.

A Novel Historical Landslide Detection Approach Based on LiDAR and Lightweight Attention U-Net

Rapid and accurate identification of landslides is an essential part of landslide hazard assessment, and in particular it is useful for land use planning, disaster prevention, and risk control. Recent alternatives to manual landslide mapping are moving in the direction of artificial intelligence—aided recognition of these surface processes. However, so far, the technological advancements have not produced robust automated mapping tools whose domain of validity holds in any area across the globe. For instance, capturing historical landslides in densely vegetated areas is still a challenge. This study proposed a deep learning method based on Light Detection and Ranging (LiDAR) data for automatic identification of historical landslides. Additionally, it tested this method in the Jiuzhaigou earthquake-hit region of Sichuan Province (China). Specifically, we generated a Red Relief Image Map (RRIM), which was obtained via high-precision airborne LiDAR data, and on the basis of this information we trained a Lightweight Attention U-Net (LAU-Net) to map a total of 1949 historical landslides. Overall, our model recognized the aforementioned landslides with high accuracy and relatively low computational costs. We compared multiple performance indexes across several deep learning routines and different data types. The results showed that the Multiple-Class based Semantic Image Segmentation (MIOU) and the F1_score of the LAU-Net and RRIM reached 82.29% and 87.45%, which represented the best performance among the methods we tested.

Foreign Object Detection in Railway Images Based on an Efficient Two-Stage Convolutional Neural Network.

Foreign object intrusion is one of the main causes of train accidents that threaten human life and public property. Thus, the real-time detection of foreign objects intruding on the railway is important to prevent the train from colliding with foreign objects. Currently, the detection of railway foreign objects is mainly performed manually, which is prone to negligence and inefficient. In this study, an efficient two-stage framework is proposed for foreign object detection in railway images. In the first stage, a lightweight railway image classification network is established to classify any input railway images into one of two classes: normal or intruded. To enable real-time and accurate classification, we propose an improved inverted residual unit by introducing two improvements to the original inverted residual unit. First, the selective kernel convolution is used to dynamically select kernel size and learn multiscale features from railway images. Second, we employ a lightweight attention mechanism, called the convolutional block attention module, to exploit both spatial and channel-wise relationships between feature maps. In the second stage of our framework, the intruded image is fed to the foreign object detection network to further detect the location and class of the objects in the image. Experimental results confirm that the performance of our classification network is comparable to the widely used baselines, and it obtains outperforming efficiency. Moreover, the performances of the second-stage object detection are satisfying.

Multiscale Feature Fusion Attention Lightweight Facial Expression Recognition

Facial expression recognition based on residual networks is important for technologies related to space human-robot interaction and collaboration but suffers from low accuracy and slow computation in complex network structures. To solve these problems, this paper proposes a multiscale feature fusion attention lightweight wide residual network. The network first uses an improved random erasing method to preprocess facial expression images, which improves the generalizability of the model. The use of a modified depthwise separable convolution in the feature extraction network reduces the computational effort associated with the network parameters and enhances the characterization of the extracted features through a channel shuffle operation. Then, an improved bottleneck block is used to reduce the dimensionality of the upper layer network feature map to further reduce the number of network parameters while enhancing the network feature extraction capability. Finally, an optimized multiscale feature lightweight attention mechanism module is embedded to further improve the feature extractability of the network for human facial expressions. The experimental results show that the accuracy of the model is 73.21%, 98.72%, and 95.21% on FER2013, CK+ and JAFFE, respectively, with a covariance of 10.14 M. Compared with other networks, the model proposed in this paper has faster computing speed and better accuracy at the same time.

ConvUNeXt: An efficient convolution neural network for medical image segmentation

Recently, ConvNeXts constructing from standard ConvNet modules has produced competitive performance in various image applications. In this paper, an efficient model based on the classical UNet, which can achieve promising results with a low number of parameters, is proposed for medical image segmentation. Inspired by ConvNeXt, the designed model is called ConvUNeXt and towards reduction in the amount of parameters while retaining outstanding segmentation superiority. Specifically, we firstly improved the convolution block of UNet by using large convolution kernels and depth-wise separable convolution to considerably decrease the number of parameters; then residual connections in both encoder and decoder are added and pooling is abandoned via adopting convolution for down-sampling; during skip connection, a lightweight attention mechanism is designed to filter out noise in low-level semantic information and suppress irrelevant features, so that the network can pay more attention to the target area. Compared to the standard UNet, our model has 20% fewer parameters, meanwhile, experimental results on different datasets show that it exhibits superior segmentation performance when the amount of data is scarce or sufficient. Code will be available at https://github.com/1914669687/ConvUNeXt.

Lightweight target detection algorithm based on improved YOLOv4

Traditional YOLOv4 object detection network is difficult to be applied to mobile embedded devices because it has some deficiencies such as complex structure and too many parameters. In this paper, the authors propose a lightweight object detection algorithm. Firstly, Mobienetv3 is used to replace the original feature extraction network, and the Mish function is used to be the activation function, which reduces the number of model parameters. Secondly, the dilated convolution is used to replace the maximum pooling operation in the original spatial pyramid pooling (SPP) structure. Then, a custom Dcn-Dw structure is used to replace the convolution operation in the original PANet, which improves the accuracy of the model for irregular object detection and reduces the model size. Finally, a CBAM lightweight attention mechanism module is introduced in front of the YOLO Head, which further improves the model accuracy. An experiment on the VOC2007 dataset is carried out, and the results show that the mean average precision (mAP) is 80.3% and frames per second (FPS) is 15. At the expense of 3% accuracy, the detection speed is increased to two to three times, and the model size is reduced to one-fifth of the original model. The lightweight object detection algorithm can suit for real-time detection tasks on resource-constrained embedded devices.

Molormer: a lightweight self-attention-based method focused on spatial structure of molecular graph for drug-drug interactions prediction.

Multi-drug combinations for the treatment of complex diseases are gradually becoming an important treatment, and this type of treatment can take advantage of the synergistic effects among drugs. However, drug-drug interactions (DDIs) are not just all beneficial. Accurate and rapid identifications of the DDIs are essential to enhance the effectiveness of combination therapy and avoid unintended side effects. Traditional DDIs prediction methods use only drug sequence information or drug graph information, which ignores information about the position of atoms and edges in the spatial structure. In this paper, we propose Molormer, a method based on a lightweight attention mechanism for DDIs prediction. Molormer takes the two-dimension (2D) structures of drugs as input and encodes the molecular graph with spatial information. Besides, Molormer uses lightweight-based attention mechanism and self-attention distilling to process spatially the encoded molecular graph, which not only retains the multi-headed attention mechanism but also reduces the computational and storage costs. Finally, we use the Siamese network architecture to serve as the architecture of Molormer, which can make full use of the limited data to train the model for better performance and also limit the differences to some extent between networks dealing with drug features. Experiments show that our proposed method outperforms state-of-the-art methods in Accuracy, Precision, Recall and F1 on multi-label DDIs dataset. In the case study section, we used Molormer to make predictions of new interactions for the drugs Aliskiren, Selexipag and Vorapaxar and validated parts of the predictions. Code and models are available at https://github.com/IsXudongZhang/Molormer.

Dimension-aware attention for efficient mobile networks

Recently, attention mechanisms have shown great potential in improving the performance of mobile networks. Typically, they involve 2D symmetric convolution operations or generate 2D attention maps. However, such manners usually introduce high computational cost and large memory consumption, increasing the computational burden of mobile networks. To address this problem, we propose a novel lightweight attention mechanism, called Dimension-Aware Attention (DAA) block, by modeling the intra-dependencies of each dimension of the input feature map. Specifically, we factorize the channel and spatial attention by three parallel feature vector encoding branches, where stacked 1D asymmetric convolution operations can be naturally leveraged to capture large receptive fields. In this way, channel-aware, horizontal-aware, and vertical-aware attention vectors are extracted to effectively encode multi-dimensional information and greatly reduce the computational complexity of mobile networks. Experiments on multiple vision tasks demonstrate that our DAA block achieves better accuracy against state-of-the-art attention mechanisms with much lower computational operations. Our code is available at https://github.com/rymo96/DAANet.

Self-supervised coarse-to-fine monocular depth estimation using a lightweight attention module

Self-supervised monocular depth estimation has been widely investigated and applied in previous works. However, existing methods suffer from texture-copy, depth drift, and incomplete structure. It is difficult for normal CNN networks to completely understand the relationship between the object and its surrounding environment. Moreover, it is hard to design the depth smoothness loss to balance depth smoothness and sharpness. To address these issues, we propose a coarse-to-fine method with a normalized convolutional block attention module (NCBAM). In the coarse estimation stage, we incorporate the NCBAM into depth and pose networks to overcome the texture-copy and depth drift problems. Then, we use a new network to refine the coarse depth guided by the color image and produce a structure-preserving depth result in the refinement stage. Our method can produce results competitive with state-of-the-art methods. Comprehensive experiments prove the effectiveness of our two-stage method using the NCBAM.

FGAM: A pluggable light-weight attention module for medical image segmentation

Medical image segmentation is fundamental for computer-aided diagnosis or surgery. Various attention modules are proposed to improve segmentation results, which exist some limitations for medical image segmentation, such as large computations, weak framework applicability, etc. To solve the problems, we propose a new attention module named FGAM, short for Feature Guided Attention Module, which is a simple but pluggable and effective module for medical image segmentation. The FGAM tries to dig out the feature representation ability in the encoder and decoder features. Specifically, the decoder shallow layer always contains abundant information, which is taken as a queryable feature dictionary in the FGAM. The module contains a parameter-free activator and can be deleted after various encoder-decoder networks’ training. The efficacy of the FGAM is proved on various encoder-decoder models based on five datasets, including four publicly available datasets and one in-house dataset.

Intelligent garbage classification system based on improve MobileNetV3-Large

In response to the call for implementing national waste classification, this paper proposes an intelligent waste classification system based on the improved MobileNetV3-Large, which can raise the national awareness of waste classification through the combination of software and hardware. The software module is based on WeChat applet and offers functions for image recognition, text recognition, speech recognition, points-based quiz and so on. The hardware module is based on Raspberry Pi and covers image shooting, image recognition, automatic classification with automatic announcement and so on. The algorithm model applied to the image classification adopts a network model based on MobileNetV3-Large. This network model is enabled to classify garbage images through deep separable convolution, inverse residual structure, lightweight attention structure and the hard_ swish activation function. The text classification model adopts a network model based on LSTM, extracts text features through word embedding, enhancing the effect of garbage text classification. After testing, the system can leverage deep learning to realise intelligent garbage classification. The image recognition accuracy of the algorithm model was found to reach 81%, while the text recognition accuracy was as high as 97.61%.

An efficient attention module for instance segmentation network in pest monitoring

• Designed a lightweight attention module to focus on the informative features. • The optimized SR method improved detection and segmentation performances. • Provided a pest dataset that contains 9,872 images and corresponding annotations. • The proposed network outperformed previous studies for pest localization. Prompt and specialized pest management involving localization and recognition has become a crucial means to prevent pest attacks in modern agriculture. Traditional pest monitoring methods are inaccurate and inefficient due to the hand-crafted features and the low-resolution images. As a result, this study presents an automatic framework that can precisely detect 10 species of pests in the natural environment and assist humans in identifying the locations and contours of pests efficiently. The main contributions of this paper include (1) a novel attention module that encourages the network to focus on the important features; (2) an optimized super-resolution approach that is used for both training and testing images to enhance the image quality; (3) a pest monitoring network is proposed by improving the D2Det’s structure and adjusting the parameters; and (4) a dataset containing pest images and manually annotated files. Experiments showed that the proposed Pest-D2Det model achieved state-of-the-art performances in terms of the mean Average Precision (mAP) values of detection (78.6%) and segmentation (77.2%). Meanwhile, the performance of our efficient channel and spatial attention network (ECSA-Net) indicated that it is lightweight and effective, which can be integrated into deep learning-based models without computation burden.

On restoration of degraded fingerprints

The state-of-the-art fingerprint matching systems achieve high accuracy on good quality fingerprints. However, degraded fingerprints obtained due to poor skin conditions of subjects or fingerprints obtained around a crime scene often have noisy background and poor ridge structure. Such degraded fingerprints pose problem for the existing fingerprint recognition systems. This paper presents a fingerprint restoration model for a poor quality fingerprint that reconstructs a binarized fingerprint image with an improved ridge structure. In particular, we demonstrate the effectiveness of channel refinement in fingerprint restoration. The state-of-the-art channel refinement mechanisms, such as Squeeze and Excitation (SE) block, in general, create SE- block introduce redundancy among channel weights and degrade the performance of fingerprint enhancement models. We present a lightweight attention mechanism that performs channel refinement by reducing redundancy among channel weights of the convolutional kernels. Restored fingerprints generated after introducing proposed channel refinement unit obtain improved quality scores on standard fingerprint quality assessment tool. Furthermore, restored fingerprints achieve improved fingerprint matching performance. We also illustrate that the idea of introducing a channel refinement unit is generalizable to different deep architectures. Additionally, to quantify the ridge preservation ability of the model, standard metrics: Dice score, Jaccard Similarity, SSIM and PSNR are computed with the ground truth and the output of the model (CR-GAN). An ablation study is conducted to individually quantify the improvement of generator and discriminator sub-networks of CR-GAN through channel refinement. Experiments on the publicly available IIITD- MOLF, Rural Indian Fingerprint Database and a private rural fingerprint database demonstrate the efficacy of the proposed attention mechanism.

Efficient Lane Detection Technique Based on Lightweight Attention Deep Neural Network

For self-driving vehicles, detecting lane lines in changeable scenarios is a fundamental yet challenging task. The rise of deep learning in recent years has contributed to the thriving of autonomous driving. However, existing methods of lane detection based on deep learning have high requirements on computing environment, so their applicability is further restricted. This paper proposed an improved attention deep neural network (DNN), a lightweight semantic segmentation architecture catering for efficient computation in low memory, which contains two branches worked in different resolution. The proposed network integrates fine details captured by local interaction of pixels at high resolution into global contexts at low resolution, computing dense feature maps for prediction task. Based on the attributes of disparate feature resolution characteristics, different attention mechanisms are adopted to guide the network to effectively exploit the model parameters. The proposed network achieves comparable results with state-of-the-art methods on two popular lane detection benchmarks (TuSimple and CULane), with faster calculation efficiency at 259 frames-per-second (FPS) on CULane dataset, and the total number of model parameters only requires 1.57 M. This study provides a practical and meaningful reference for the application of lane detection in memory constrained devices.