Output Feature Maps Research Articles

LED Chip Defect Detection Method Based on a Hybrid Algorithm

LED is an extremely important energy-saving lighting products, which has greatly facilitated human life. Meanwhile, it also makes a positive contribution to global carbon neutrality and carbon peaking. Defect detection is a vital part of the production process to control the quality of LED chips. The traditional methods use a microscope for manual visual inspection, which is time-consuming and has inconsistent testing standards, low efficiency, and other deficiencies. To solve these problems, a hybrid algorithm based on geometric computation and a convolutional neural network is proposed for LED chip defect detection. The method takes advantage of the dimensionality reduction of geometric computation to perform coarse detection of defects on preprocessed chip lithography graphs in the form of grid segmentation, which realizes fast coarse screening of large-scale chip samples and reduces postcomputational costs. The convolutional neural network model is used for the secondary fine detection of “suspected defective” chips after geometric coarse screening, and the SPP (spatial pyramid pooling) network model is improved by directly introducing the original feature map into the SPP pooling layer for summation to enhance the global and local feature information of the output feature map. Furthermore, we construct an LED chip image acquisition platform using a high-frequency multimagnification zoom lens, collect training samples of defective chips, and increase the number of samples through image processing techniques. The research introduces the R-CNN, SDD, and YOLO methods to evaluate the superiority of our method in a number of experiments. The experimental results show that our algorithm proposed in this paper has an average precision (AP) of 96.7% for large-scale chip detection with a low defect rate. Compared with other methods, the testing mean average precision (mAP) is 10.39% higher than traditional YOLO v2. The testing mIoU is also 3.63% higher than traditional YOLO v5, the detection speed is also significantly improved, and it has good robustness.

A classification model for detection of ductal carcinoma in situ by Fourier transform infrared spectroscopy based on deep structured semantic model

At present, deep learning is widely used in spectral data processing. Deep learning requires a large amount of data for training, while the collection of biological serum spectra is limited by sample numbers and labor costs, so it is impractical to obtain a large amount of serum spectral data for disease detection. In this study, we propose a spectral classification model based on the deep structured semantic model (DSSM) and successfully apply it to Fourier Transform Infrared (FT-IR) spectroscopy for ductal carcinoma in situ (DCIS) detection. Compared with the traditional deep learning model, we match the spectral data into positive and negative pairs according to whether the spectra are from the same category. The DSSM structure is constructed by extracting features according to the spectral similarity of spectra pairs. This new construction model increases the data amount used for model training and reduces the dimension of spectral data. Firstly, the FT-IR spectra are paired. The spectra pairs are labeled as positive pairs if they come from the same category, and the spectra pairs are labeled as negative pairs if they come from different categories. Secondly, two spectra in each spectra pair are put into two deep neural networks of the DSSM structure separately. Then the spectral similarity between the output feature maps of two deep neural networks is calculated. The DSSM structure is trained by maximizing the conditional likelihood of the spectra pairs from the same category. Thirdly, after the training of DSSM is done, the training set and testing set are input into two deep neural networks separately. The output feature maps of the training set are put into the reference library. Lastly, the k-nearest neighbor (KNN) model is used for classification according to Euclidean distances between the output feature map of each unknown sample to the reference library. The category of the unknown sample is judged according to the categories of k nearest samples. We also use principal component analysis (PCA) to reduce dimension for comparison. The accuracies of the KNN model, principal component analysis-k nearest neighbor (PCA-KNN) model, and deep structured semantic model-k nearest neighbor (DSSM-KNN) model are 78.8%, 72.7%, and 97.0%, which proves that our proposed model has higher accuracy.

A long-memory pedestrian target tracking algorithm incorporating spatiotemporal trajectory feature enhancement model

Multi-target tracking is a complex problem in computer vision, involving occlusion, target trajectory disappearance, etc. In response to these questions, we first propose a trajectory coordinate bi-directional sensing attention module (BasicCaModule). This module enables the network to be more focused on essential areas. Secondly, we propose the Feature Confusion Edge Enhancement (FCEE) module, which enhances the confusion region of adjacent feature edges. After detecting the network output feature map, we propose the adaptive feature separation header (FDH) module. This module can activate essential features adaptively to reduce the number of target identity switches. Finally, appropriate hyperparameters are set and optimized for the ByteTrack tracking algorithm, resolving fragment tracking loss and long memory matching problems. After a large amount of experimental data, our approach is highly competitive with the existing state-of-the-art algorithms. Moreover, it can also keep online tracking while maintaining high tracking accuracy. The code is publicly available at: https://github.com/CccJ23/Spatio-temporal-trajectory-Tracking

Long-Term Energy Forecasting System Based on LSTM and Deep Extreme Machine Learning

Due to the development of diversified and flexible building energy resources, the balancing energy supply and demand especially in smart buildings caused an increasing problem. Energy forecasting is necessary to address building energy issues and comfort challenges that drive urbanization and consequent increases in energy consumption. Recently, their management has a great significance as resources become scarcer and their emissions increase. In this article, we propose an intelligent energy forecasting method based on hybrid deep learning, in which the data collected by the smart home through meters is put into the pre-evaluation step. Next, the refined data is the input of a Long Short-Term Memory (LSTM) network, which captures the spatio-temporal correlations from the sequence and generates the feature maps. The output feature map is passed into a Deep Extreme Machine Learning network (with seven hidden layers) for learning, which provides the final prediction. Compared to existing techniques, the LSTM-DELM model offers better prediction results. The simulation values demonstrate the superior performance of the proposed model.

StAIn: Stealthy Avenues of Attacks on Horizontally Collaborated Convolutional Neural Network Inference and Their Mitigation

With significant potential improvement in device-to-device (D2D) communication due to improved wireless link capacity (e.g., 5G and NextG systems), a collaboration of multiple edge devices (called horizontal collaboration (HC)) is becoming a reality for real-time Edge Intelligence (EI). The distributed nature of HC offers an advantage against traditional adversarial attacks because the adversary does not have access to the entire deep learning architecture (DLA). Due to the involvement of multiple untrusted edge devices in HC environment, the possibility of malicious devices cannot be eliminated. In this paper, we unearth some attacks that are very effective and stealthy even when the attacker has minimal knowledge of the DLA as is the case in HC-based DLA. We are also providing novel filtering methods to mitigate such attacks. Our novel attacks leverage local information available on output feature maps (FMs) of a targeted edge device to modify the regular adversarial attacks (e.g. Fast Gradient Signed Method (FGSM) and Jacobian-based Saliency Map Attack (JSMA)). Similarly, a customized convolutional neural network (CNN) based filter is empirically designed, developed, and tested. Four different CNN models (LeNet, CapsuleNet, MiniVGGNet, and VGG16) are used to validate the proposed attacks and defense methodologies. Our three attacks on four different CNN models (with two variations of each attack) show a substantial accuracy drop of 62% on average. The proposed filtering approach is able to mitigate the attack by recovering the actual accuracy back to 75.1% on average. To the best of our knowledge, this is the first work that investigates the security vulnerability of DLA in the HC environment, and all three of our attacks are scalable and agnostic to the partition location within the DLA.

DSFNet: A Distributed Sensors Fusion Network for Action Recognition

Human action recognition (HAR) has become a hot topic in the field of computer vision and pattern recognition due to its wide range of application prospects. In most deep learning and multisensor data-based action recognition works, sequences from sensors are first stacked into 2-D images and then fed into convolutional neural network (CNN) models to extract features. This approach based on data-level fusion learns different sensor data features but loses the uniqueness of different sensor data. In this article, we propose a new deep learning model called distributed sensors fusion network (DSFNet). For the property that acceleration sequences can reflect motion trends, we first learn the features of different sensors and then use the Transformer encoder module to model the dependencies of multisensor actions and extract features. For the property that angular velocity can reflect the direction and velocity of the local pose, we first learn the features of a single sensor in different motion directions in time sequence and then learn the output feature maps of multisensor features in time sequence and extract the features. Finally, the outputs of different data modalities are used for decision-level fusion, which significantly improves the performance of the model. We evaluate the performance of the proposed model on the self-built dataset Changzhou University: a comprehensive multi-modal human action dataset (CZU-MHAD), and the experimental results show that the DSFNet model outperforms the existing methods.

PFYOLOv4: An Improved Small Object Pedestrian Detection Algorithm

With the development of deep convolutional neural networks, the effect of pedestrian detection has been rapidly improved. However, there are still many problems in small target pedestrian detection, for example noise (such as light) interference, target occlusion, and low detection accuracy. In order to solve the above problems, based on YOLOv4 algorithm, this paper proposes an improved small target pedestrian detection algorithm named PF_YOLOv4. The algorithm is improved in three aspects on the basis of the YOLOv4 algorithm: firstly, a soft thresholding module is added to the residual structure of the backbone network to perform noise reduction process on interference factors, such as light to enhance the robustness of the algorithm; secondly, the depthwise separable convolution replaces the traditional convolution in the YOLOv4 residual structure, to reduce the number of network model parameters; finally, the Convolutional Block Attention Module (CBAM) is added after the output feature map of the backbone network to enhance of the network feature expression. Experimental results show that the PF_YOLOv4 algorithm outperforms most of the state-of-the-art algorithms in detecting small target pedestrians. The mean Average Precision (mAP) of the PF_YOLOv4 algorithm is 2.35% higher than that of the YOLOv4 algorithm and 9.67% higher than that of the YOLOv3 algorithm, while the detection speed is slightly higher than that of YOLOv4 algorithm.

A graphical approach for filter pruning by exploring the similarity relation between feature maps

The vast majority of repetitive pruning and retraining techniques used on CNNs require multi-stage optimization, which undermines the potential computing savings from pruning. The similarity relationship between the output feature maps of the filters is first observed and represented as a graphical model in this paper in order to solve this issue. Vertex redundancy in a graph is determined using the degree of a vertex and the weight of its edges. Pruning is then carried out “one-shot” by removing the filters associated with redundant feature maps from each layer. In order to preserve performance, the network must finally be retrained. The CIFAR-10 dataset is identified using the classifier produced by applying this proposed methodology to several deep learning models. In this instance, both the parameters reduction and FLOPs reduction of the trimmed model are significantly improved. This method, tested on VGG-16, reduces parameters by 90.8% and FLOPs by 60.0% while sacrificing little accuracy (=0.16%) in comparison to the baseline. The results based on ResNet-110 show that applying the proposed approach can reduce FLOPs and parameters by 70.4% while preserving baseline accuracy with a loss of just 0.05%.

Spatio-Temporal Human Action Recognition Model using Deep Learning Techniques

Two-stream human recognition achieved great success in the development of video action recognition using deep learning. Recently many studies have shown that two-stream action recognition is a powerful feature extractor. The main contribution in this work is to develop a two-stream model based on spatial and temporal networks using convolutional neural networks with a convolution long-short term memory. The two-stream model with ImageNet pre-trained weights is used to retrieve spatial and temporal features. Output feature maps of the two-stream model are fused using sum fusion and fed as input to convolutional long-short-term memory. SoftMax function is used to get the final classification score. To avoid overfitting, we have adopted the data augmentation techniques. Finally, we demonstrated that the proposed model performs well in comparison to state-of-the-art two-stream models with an accuracy of 96.1% on UCF 101 dataset and 70.9% accuracy on the HMDB dataset.

Strong Generalized Speech Emotion Recognition Based on Effective Data Augmentation

The absence of labeled samples limits the development of speech emotion recognition (SER). Data augmentation is an effective way to address sample sparsity. However, there is a lack of research on data augmentation algorithms in the field of SER. In this paper, the effectiveness of classical acoustic data augmentation methods in SER is analyzed, based on which a strong generalized speech emotion recognition model based on effective data augmentation is proposed. The model uses a multi-channel feature extractor consisting of multiple sub-networks to extract emotional representations. Different kinds of augmented data that can effectively improve SER performance are fed into the sub-networks, and the emotional representations are obtained by the weighted fusion of the output feature maps of each sub-network. And in order to make the model robust to unseen speakers, we employ adversarial training to generalize emotion representations. A discriminator is used to estimate the Wasserstein distance between the feature distributions of different speakers and to force the feature extractor to learn the speaker-invariant emotional representations by adversarial training. The simulation experimental results on the IEMOCAP corpus show that the performance of the proposed method is 2-9% ahead of the related SER algorithm, which proves the effectiveness of the proposed method.

Improving Deep Learning-based Plant Disease Classification with Attention Mechanism

In recent years, deep learning-based plant disease classification has been widely developed. However, it is challenging to collect sufficient annotated image data to effectively train deep learning models for plant disease recognition. The attention mechanism in deep learning assists the model to focus on the informative data segments and extract the discriminative features of inputs to enhance training performance. This paper investigates the Convolutional Block Attention Module (CBAM) to improve classification with CNNs, which is a lightweight attention module that can be plugged into any CNN architecture with negligible overhead. Specifically, CBAM is applied to the output feature map of CNNs to highlight important local regions and extract more discriminative features. Well-known CNN models (i.e. EfficientNetB0, MobileNetV2, ResNet50, InceptionV3, and VGG19) were applied to do transfer learning for plant disease classification and then fine-tuned by a publicly available plant disease dataset of foliar diseases in pear trees called DiaMOS Plant. Amongst others, this dataset contains 3006 images of leaves affected by different stress symptoms. Among the tested CNNs, EfficientNetB0 has shown the best performance. EfficientNetB0+CBAM has outperformed EfficientNetB0 and obtained 86.89% classification accuracy. Experimental results show the effectiveness of the attention mechanism to improve the recognition accuracy of pre-trained CNNs when there are few training data.

An Energy-Efficient Convolutional Neural Network Processor Architecture Based on a Systolic Array

Deep convolutional neural networks (CNNs) have shown strong abilities in the application of artificial intelligence. However, due to their extensive amount of computation, traditional processors have low energy efficiency when executing CNN algorithms, which is unacceptable for portable devices with limited hardware cost and battery capacity, so designing a CNN-specific processor is necessary. In this paper, we propose an energy-efficient CNN processor architecture for lightweight devices with a processing elements (PEs) array consisting of 384 PEs. Using the systolic array-based PE array, it realizes parallel operations between filter rows and between channels of output feature maps, supporting the acceleration of 3D convolution and fully connected computation with various parameters by configuring internal instruction registers. The computing strategy based on the proposed systolic dataflow achieves less hardware overhead compared with other strategies, and the reuse of image values and weight values, which effectively reduce the power of memory access. A memory system with a multi-level storage structure combined with register file (RF) and SRAM is used in the proposed CNN processor, which further reduces the energy overhead of computing. The proposed CNN processor architecture has been verified on a ZC706 FPGA platform using VGG-16 based on the proposed image segmentation method, the evaluation results indicate that the peak throughput achieves 115.2 GOP/s consuming 3.801 W at 150 MHz, energy efficiency and DSP efficiency reaches 30.32 GOP/s/W and 0.26 GOP/s/DSP, respectively.

Cell Phenotype Classification Based on Joint of Texture Information and Multilayer Feature Extraction in DenseNet.

Cell phenotype classification is a critical task in many medical applications, such as protein localization, gene effect identification, and cancer diagnosis in some types. Fluorescence imaging is the most efficient tool to analyze the biological characteristics of cells. So cell phenotype classification in fluorescence microscopy images has received increased attention from scientists in the last decade. The visible structures of cells are usually different in terms of shape, texture, relationship between intensities, etc. In this scope, most of the presented approaches use one type or joint of low-level and high-level features. In this paper, a new approach is proposed based on a combination of low-level and high-level features. An improved version of local quinary patterns is used to extract low-level texture features. Also, an innovative multilayer deep feature extraction method is performed to extract high-level features from DenseNet. In this respect, an output feature map of dense blocks is entered in a separate way to pooling and flatten layers, and finally, feature vectors are concatenated. The performance of the proposed approach is evaluated on the benchmark dataset 2D-HeLa in terms of accuracy. Also, the proposed approach is compared with state-of-the-art methods in terms of classification accuracy. Comparison of results demonstrates higher performance of the proposed approach in comparison with some efficient methods.

High Speed Railway Fastener Defect Detection by Using Improved YoLoX-Nano Model

Rails play a vital role in the bearing and guidance of high-speed trains, and the normal condition of rail components is the guarantee of the operation and maintenance safety. Fasteners are critical components for fixing the rails, so it is particularly important to detect whether they are in a normal state or not. The current rail-fastener detection models have some drawbacks, including poor generalization ability, large model volume and low detection efficiency. In view of this, an improved YoLoX-Nano rail-fastener-defect-detection method is proposed in this paper. The CA attention mechanism is added to the three output feature maps of CSPDarknet and the enhanced feature extraction part of the Path Aggregation Feature Pyramid Network (PAFPN); the Adaptively Spatial Feature Fusion (ASFF) is added after the PAFPN output feature map, which enables the semantic information of the high-level features and the fine-grained features of the bottom layer to be further enhanced. The improved YoLoX-Nano model has improved the AP value by 27.42% on fractured fasteners, 15.88% on displacement fasteners and 12.96% on normal fasteners. Moreover, the mAP value is improved by 18.75%, and it is 14.75% higher than the two-stage model Faster-RCNN on mAP. In addition, compared with YoLov7-tiny, the improved YoLoX-Nano model achieves 13.56% improvement on mAP. Although the improved model increases a certain amount of calculation, the detection speed of the improved model has been increased by 30.54 fps and by 32.33 fps when compared with that of the Single-Shot Multi-Box Detector (SSD) model and the You Only Look Once v3 (YoLov3) model, reaching 54.35 fps. The improved YoLoX-Nano model enables accurate and rapid identification of the defects of rail fasteners, which can meet the needs of real-time detection. Furthermore, it has advantages in lightweight deployment of terminals for rail-fastener detection, thus providing some reference for image recognition and detection in other fields.

Binary residual feature pyramid network: An improved feature fusion module based on double‐channel residual pyramid structure for autonomous detection algorithm

AbstractThe vehicle detection algorithm based on visual perception has been applied in all types of automatic driving scenes. However, there are still flaws in the current detection algorithm model, especially for small objects. The detection effect of vehicle objects with small pixels in the image is often missed and wrongly detected. This research proposes an improved feature fusion module based on double‐channel residual pyramid (DRP) structure for autonomous detection algorithm which named binary residual feature pyramid network (BiResFPN) to solve the above problems. Firstly, a DRP structure, which can effectively supplement the shallow information of the network, is proposed. The residual structure is added to the output feature map for further supplement. Then, an average sampling method of positive and negative samples based on intersection‐over‐union (IOU) value is proposed on the basis of this structure, aimed at the unbalanced sampling of positive and negative samples in the training stage of faster regions with CNN features (RCNN). It leads to the reduction of the interference of a large number of simple negative samples, which makes the learned model better. The experimental results based on the KITTI and BDD100K dataset datasets show that the capability of the feature fusion module based on DRP structure is strong for small object detection. Compared with Faster‐RCNN (FPN), the detection algorithm of small object detection accuracy APsmall was increased by 2.6%, APmedium and APlarge was increased by 1.1% and 0.3%.

Remote sensing aircraft object detection algorithm based on memory CenterNet

In order to solve the problem of poor detection accuracy of CenterNet in remote sensing aircraft with dense targets and complex backgrounds, an improved memory CenterNet object detection algorithm is proposed. First, the multiscale receptive field module and residual attention module are introduced to improve ResNet50, which is used as a new backbone feature extraction network. It has a strong feature extraction ability and detection speed at the same time. Different dilated ratios are used in the multiscale receptive field to expand the receptive field and extract multiscale features, and the multiscale target features are highlighted through the attention mechanism. Then the last set of residual networks in ResNet50 is deleted, and the residual attention network is introduced and stacked in the same way. The feature parameters of different channels in the residual attention network are calculated by two-dimensional cosine transform and weighted with the original feature map, and the target features are highlighted to enhance the ability to locate the target at the end of the network. Second, three times deconvolution is replaced by a memory feature fusion module. The shallow detail information and the deep semantic information are fully fused. The key information in the output feature map is further enriched, which is beneficial to the improvement of model detection performance. Finally, the experimental results show that the average accuracy of remote sensing aircraft dataset is 88.46%, which is 18.21% higher than that of the original algorithm and effectively improves the detection accuracy of remote sensing aircraft targets.

Neural network pruning based on channel attention mechanism

Network pruning facilitates the deployment of convolutional neural networks in resource-limited environments by reducing redundant parameters. However, most of the existing methods ignore the differences in the contributions of the output feature maps. In response to the above, we propose a novel neural network pruning method based on the channel attention mechanism. In this paper, we firstly utilise the principal component analysis algorithm to reduce the influence of noisy data on feature maps. Then, we propose an improved Leaky-Squeeze-and-Excitation block to evaluate the contribution of each output feature map using the channel attention mechanism. Finally, we effectively remove lower contribution channels without reducing the model performance as much as possible. Extensive experimental results show that our proposed method achieves significant improvements over the state-of-the-art in terms of FLOPs and parameters reduction with similar accuracies. For example, with VGG-16-baseline, our proposed method reduces parameters by 83.3% and FLOPs by 66.3%, with only a loss of 0.13% in top-5 accuracy. Furthermore, it effectively balances pruning efficiency and prediction accuracy.

Capitalizing on RGB-FIR Hybrid Imaging for Road Detection

Traditionally, road detection approaches mostly capitalize on RGB images, 3D LiDAR point cloud or their fusion. However, RGB camera is sensitive to light conditions, while LiDAR point cloud is sparse compared with dense image pixels. In this work, a new hybrid image dataset is provided for the task of road detection based on cameras. In this dataset, the hybrid images are acquired by an optically aligned hybrid imaging device, consisting of a far-infrared (FIR) imager and an RGB camera to output pixel-wise registration of thermal and RGB frames. Then we investigate on three methods based on fully convolutional neural network (F-CNN) to demonstrate the advantages by fusing RGB-FIR images in road detection. First, a middle-fusion based model is built, where the output feature maps of encoder branches from RGB and FIR images are directly concatenated into a single-fusion branch as the decoder. Next, the originally discarded layers after fusion operation for both RGB and FIR branches are recovered as the mimic branches to imitate the distributions of the fusion outputs, which constitutes an extended cross model (ECM). Moreover, the outputs of mimic branches at different scales are also used to imitate the corresponding outputs in the fusion branch, called a hierarchical cross model (HCM). The experimental results demonstrate the effectiveness and efficiency of our fusion strategies.

A multi-branch hierarchical attention network for medical target segmentation

Medical imaging techniques have been widely used in modern clinical disease diagnosis and treatment programs. The captured medical images can well reflect the conditions of the human body tissues, which are significantly helpful to the doctors to determine the existence or the severity of the disease. In this paper, we develop a hierarchical attentive high-resolution convolutional network (AttHRNet) for segmenting targets of interest from medical images aiming to improve the automated processing standard and the intelligent interpretation quality of the medical images. The AttHRNet is an improved version of the high-resolution network (HRNet) structure with three novel modules. First, built with an improved HRNet structure assisted by a multiscale context augmentation (MSCA) module as the feature extraction backbone, the AttHRNet can produce a set of high-quality, strong-semantic feature maps at different resolutions. The MSCA module functions to reduce the information loss during feature downsampling. Second, designed with an effective feature attention principle, the feature encoding quality in each branch can be significantly promoted by concentrating on the informative and salient feature encodings across both channels and spatial locations. Furthermore, formulated with a hierarchical segmentation scheme, the output feature maps can be further augmented by including the semantic-level category exploitation (SLCE) module with a global perspective. The SLCE module allows the information from lower resolution segmentations to inform higher resolution segmentations. Through quantitative examinations, visual verifications, and comparative evaluations on four medical image datasets, we convince the promising applicability and competitive superiority of the AttHRNet in medical target segmentation issues.

Unsupervised Domain Adaptation Using Adversarial Learning and Maximum Square Loss for Liver Tumors Detection in Multi-phase CT Images.

Automatic and efficient liver tumor detection in multi-phase CT images is essential in computer-aided diagnosis of liver tumors. Nowadays, deep learning has been widely used in medical applications. Normally, deep learning-based AI systems need a large quantity of training data, but in the medical field, acquiring sufficient training data with high-quality annotations is a significant challenge. To solve the lack of training data issue, domain adaptation-based methods have recently been developed as a technique to bridge the domain gap across datasets with different feature characteristics and data distributions. This paper presents a domain adaptation-based method for detecting liver tumors in multi-phase CT images. We adopt knowledge for model learning from PV phase images to ART and NC phase images. Clinical Relevance- To minimize the domain gap we employ an adversarial learning scheme with the maximum square loss for mid-level output feature maps using an anchorless detector. Experiments show that our proposed method performs much better for various CT-phase images than normal training.