Features Of Convolutional Layers Research Articles

Vehicle-Type Recognition Method for Images Based on Improved Faster R-CNN Model.

The rapid increase in the number of vehicles has led to increasing traffic congestion, traffic accidents, and motor vehicle crime rates. The management of various parking lots has also become increasingly challenging. Vehicle-type recognition technology can reduce the workload of humans in vehicle management operations. Therefore, the application of image technology for vehicle-type recognition is of great significance for integrated traffic management. In this paper, an improved faster region with convolutional neural network features (Faster R-CNN) model was proposed for vehicle-type recognition. Firstly, the output features of different convolution layers were combined to improve the recognition accuracy. Then, the average precision (AP) of the recognition model was improved through the contextual features of the original image and the object bounding box optimization strategy. Finally, the comparison experiment used the vehicle image dataset of three vehicle types, including cars, sports utility vehicles (SUVs), and vans. The experimental results show that the improved recognition model can effectively identify vehicle types in the images. The AP of the three vehicle types is 83.2%, 79.2%, and 78.4%, respectively, and the mean average precision (mAP) is 1.7% higher than that of the traditional Faster R-CNN model.

MAWKDN: A Multimodal Fusion Wavelet Knowledge Distillation Approach Based on Cross-View Attention for Action Recognition

The recognition prerformance of existing vision-based human action recognition (HAR) methods is greatly reduced in the case of low camera resolution or occlusion. Wearable sensors can provide complementary information to alleviate this problem. It is challenging to construct a robust HAR model using multimodal wearable-sensor data. In this paper, we propose a cross-Attention-based Multimodal fusion Wavelet Knowledge Distillation Network (MAWKDN) method to guide recognition from video data by acquiring complementary information from wearable sensors and reduce the noise effects through wavelet knowledge distillation, which improves the robustness of the model. A multi-attention dilated convolution kernel residual network including dilated convolution and an attention mechanism is constructed to extract features from various sensor modalities and fuse the various modal data through the cross-view attention method to acquire additional information from different modalities. To reduce the modal differences between different modalities of the teacher and student networks and acquire similar semantic knowledge, we learn the information between different modalities by constructing a graph structure of convolutional layer features, and computing the semantic preservation loss between the teacher and student networks. To reduce the influence of noise in the input data, we construct the loss of wavelet knowledge distillation, which transforms the image through the discrete wavelet transform and only retains the low frequency features to extract the useful information. The top-1 accuracy achieved on the UTD-MHAD (99.31%), Berkeley-MHAD (99.40%) and the F1-score on the MMAct (85.26% based on cross-session) dataset prove the superior performance of MAWKDN compared with the state-of-the-art HAR methods. Moreover, we demonstrate the robustness of the MAWKDN approach on the noise-added UTD-MHAD dataset.

Multiband seizure type classification based on 3D convolution with attention mechanisms

Electroencephalogram (EEG) signal contains important information about abnormal brain activity, which has become an important basis for epilepsy diagnosis. Recently, epilepsy EEG signal classification methods mainly extract features from the perspective of a single domain, which cannot effectively utilize the spatial domain information in EEG signals. The redundant information in EEG signals will affect the learning features with the increase of convolution layer and multi-domain features, resulting in inefficient learning and a lack of distinguishing features. To tackle these issues, we propose an end-to-end 3D convolutional multiband seizure-type classification model based on attention mechanisms. Specifically, to process preprocessed electroencephalogram (EEG) data, a multilevel wavelet decomposition is applied to obtain the joint distribution information in the two-dimensional time–frequency domain across multiple frequency bands. Subsequently, this information is transformed into three-dimensional spatial data based on the electrode configuration. Discriminative joint activity features in the time, frequency, and spatial domains are then extracted by a series of parallel 3D convolutional sub-networks, where 3D channels and spatial attention mechanisms improve the ability to learn critical global and local information. A multi-layer perceptron is finally implemented to integrate the extracted features and further map them to the classification results. Experimental results on the TUSZ dataset, the world’s largest publicly available seizure corpus, show that 3D-CBAMNet significantly outperforms the state-of-the-art methods, indicating effectiveness in the seizure type classification task.

A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring

The use of deep learning techniques in real-time monitoring can save a lot of manpower in various scenarios. For example, mask-wearing is an effective measure to prevent COVID-19 and other respiratory diseases, especially for vulnerable populations such as children, the elderly, and people with underlying health problems. Currently, many public places such as hospitals, nursing homes, social service facilities, and schools experiencing outbreaks require mandatory mask-wearing. However, most of the terminal devices currently available have very limited GPU capability to run large neural networks. This means that we have to keep the parameter size of a neural network modest while maintaining its performance. In this paper, we propose a framework that applies deep learning techniques to real-time monitoring and uses it for the real-time monitoring of mask-wearing status. The main contributions are as follows: First, a feature fusion technique called skip layer pooling fusion (SLPF) is proposed for image classification tasks. It fully utilizes both deep and shallow features of a convolutional neural network while minimizing the growth in model parameters caused by feature fusion. On average, this technique improves the accuracy of various neural network models by 4.78% and 5.21% on CIFAR100 and Tiny-ImageNet, respectively. Second, layer attention (LA), an attention mechanism tailor-made for feature fusion, is proposed. Since different layers of convolutional neural networks make different impacts on the final prediction results, LA learns a set of weights to better enhance the contribution of important convolutional layer features. On average, it improves the accuracy of various neural network models by 2.10% and 2.63% on CIFAR100 and Tiny-ImageNet, respectively. Third, a MobileNetv2-based lightweight mask-wearing status classification model is trained, which is suitable for deployment on mobile devices and achieves an accuracy of 95.49%. Additionally, a ResNet mask-wearing status classification model is trained, which has a larger model size but achieves high accuracy of 98.14%. By applying the proposed methods to the ResNet mask-wearing status classification model, the accuracy is improved by 1.58%. Fourth, a mask-wearing status detection model is enhanced based on YOLOv5 with a spatial-frequency fusion module resulting in a mAP improvement of 2.20%. Overall, this paper presents various techniques to improve the performance of neural networks and apply them to mask-wearing status monitoring, which can help stop pandemics.

Context-aware and part alignment for visible-infrared person re-identification

Visible-infrared person re-identification (VI-ReID) is a challenging problem of matching a person from visible and infrared modalities. Existing researches adopt the last convolutional layer features of the off-the-shelf backbone network as the representation to refine, which is unable to represent the heterogeneous cross-modality features with discriminant information. In this paper, we propose a novel graph-based aggregation learning network (GALNet) with visual Transformer embedding to mine both multi-layer features and part-level contextual cues for VI-ReID. We propose a novel feature memory module (FMM) to reserve global discriminative features of the low layers with the correlation modeling ability of the graph convolution network (GCN) which is supplemented to the final person representation. To learn more discriminant part-level features, an attentive part aggregation module (PAM) is designed to mine part relationships, leveraging the self-attention mechanism of the Transformer. By fusing these components, the global-level, and part-level discriminant information can be utilized. Extensive experiments on SYSU-MM01 and RegDB benchmarks demonstrate the effectiveness of our proposed methods compared with several state-of-the-art methods.

Classification of Appearance Quality of Red Grape Based on Transfer Learning of Convolution Neural Network

Grapes are a globally popular fruit, with grape cultivation worldwide being second only to citrus. This article focuses on the low efficiency and accuracy of traditional manual grading of red grape external appearance and proposes a small-sample red grape external appearance grading model based on transfer learning with convolutional neural networks (CNNs). In the naturally growing vineyards, 195,120,135 samples of Grade I, Grade II, and Grade III red grapes were collected using a Canon EOS 550D camera, and a data set of 1800 samples was obtained using data enhancement technology. Then, the CNN transfer learning method was used to transfer the pre-trained AlexNet, VGG16, GoogleNet, InceptionV3, and ResNet50 network models on the ImageNet image dataset to the red grape image grading task. By comparing the classification performance of the CNN models of these five different network depths with fine-tuning, ResNet50 with a learning rate of 0.001 and a loop number of 10 was determined to be the best feature extractor for red grape images. Moreover, given the small number of red grape image samples in this study, different convolutional layer features output by the ResNet50 feature extractor were analyzed layer by layer to determine the effect of deep features extracted by each convolutional layer on Support Vector Machine (SVM) classification performance. This analysis helped to obtain a ResNet50 + SVM red grape external appearance grading model based on the optimal ResNet50 feature extraction strategy. Experimental data showed that the classification model constructed using the feature parameters extracted from the 10th node of the ResNet50 network achieved an accuracy rate of 95.08% for red grape grading. These research results provide a reference for the online grading of red grape clusters based on external appearance quality and have certain guiding significance for the quality and efficiency of grape industry circulation and production.

Single-Channel Blind Image Separation Based on Transformer-Guided GAN

Blind source separation (BSS) has been a great challenge in the field of signal processing due to the unknown distribution of the source signal and the mixing matrix. Traditional methods based on statistics and information theory use prior information such as source distribution independence, non-Gaussianity, sparsity, etc. to solve this problem. Generative adversarial networks (GANs) learn source distributions through games without being constrained by statistical properties. However, the current blind image separation methods based on GANs ignores the reconstruction of the structure and details of the separated image, resulting in residual interference source information in the generated results. This paper proposes a Transformer-guided GAN guided by an attention mechanism. Through the adversarial training of the generator and the discriminator, U-shaped Network (UNet) is used to fuse the convolutional layer features to reconstruct the structure of the separated image, and Transformer is used to calculate the position attention and guide the detailed information. We validate our method with quantitative experiments, showing that it outperforms previous blind image separation algorithms in terms of PSNR and SSIM.

Adaptive spatial-temporal surrounding-aware correlation filter tracking via ensemble learning

With the advancement of computer vision, object trackers based on discriminative correlation filters (DCF) have demonstrated superior performance and accuracy compared to traditional trackers. However, most existing DCF-based trackers are easily affected by various factors, such as cluttered background, illumination variations, occlusions, rotations etc. Therefore, in order to accurately track the target, further investigation into the characteristics of the correlation filter is required. In this study, we propose an adaptive spatial-temporal surrounding-aware correlation filter tracker via the ensemble learning (ASTSAELT) technique. Specifically, the adaptive spatial-temporal regularized correlation filter to remove the boundary effects and temporal degradation is presented. And then, a method of extracting surrounding samples according to the size and shape of the tracking object, designed to preserve the integrity of the object, is proposed. Moreover, our tracker utilizes a multi-expert tracking framework to improve its performance by integrating both handcrafted features and deep convolutional layer features. And then, the update strategy is proposed to measure the reliability of the current tracking result and mitigate model corruption. Finally, numerous experiments on visual tracking benchmarks including OTB2013, OTB2015, TempleColor128, UAV123, UAVDT and DTB70 are implemented to verify the developed method achieves superior performance compared with several state-of-the-art methods.

EVALUATING DEEP LEARNING MODELS FOR MUSIC EMOTION RECOGNITION

Music listening helps people not only for entertainment, but also to reduce emotional stress in their daily lives. People nowadays tend to use online music streaming services such as Spotify, Amazon Music, Google Play Music, etc. rather than storing the songs on their devices. The songs in these streaming services are categorized into different emotional labels such as happy, sad, romantic, devotional, etc. In the music streaming applications, the songs are manually tagged with their emotional categories for music recommendation. Considering the growth of music on different social media platforms and the internet, the need for automatic tagging will increase in coming time. The work presented deals with training the deep learning model for automatic emotional tagging. It covers implementation of two different deep learning architectures for classifying the audio files using the Mel-spectrogram of music audio. The first architecture proposed is Convolutional Recurrent Model (CRNN) and the second architecture is a Parallel Convolutional Recurrent Model (Parallel CNN). Both the architectures exploit the combined features of Convolutional and Recurrent layers. This combination is used to extract features from time and frequency domains. The results with accuracies in the range of 51 to 54 % are promising for both models for a small dataset of 138 songs, considering the large datasets required for training deep learning models.

Research on Land Utilization Spatial Classification Planning Method Based on Multiocular Vision.

With the development of China's social economy as well as the accelerating urbanization construction and the expanding scale of cities, the integration of land use and urban land classification based on land use spatial planning has become an important task for the sustainable development of China at present. Land use spatial classification planning is the basic basis for all kinds of development and protection construction activities, and government land use spatial planning at all levels plays an important role in implementing major national, provincial, and municipal strategies and promoting the rational and effective use of land use space. By briefly describing the spatial classification of land use and analyzing the idea of systematic integration of land use, this paper provides guidance and reference for exploring the construction of urban land use classification under land use spatial planning, aiming to improve the classification system of land use spatial planning. A neural network-based land use classification algorithm is proposed for the problems of few labeled samples of remote sensing images with high spatial resolution and feature deformation due to sensor height changes in land use spatial classification planning. By multiscale adaptive fusion of multiple convolutional layer features, the impact of feature deformation on classification accuracy is reduced. To further improve the classification accuracy, the depth features extracted from the pretraining network are used to pretrain the multiscale feature fusion part and the fully connected layer, and the whole network is fine-tuned using the augmented dataset. The experimental results show that the adaptive fusion method improves the fusion effect and effectively improves the accuracy of land use spatial classification planning.

A Study on the Relationship between Painter's Psychology and Anime Creation Style Based on a Deep Neural Network.

We conduct an in-depth study on the model construction of deep neural networks and design a model of painter's psychology and anime creation style to realize the study of the relationship between painter's psychology and anime creation style based on deep neural networks. This paper proposes an animation creation psychology classification algorithm that integrates human cognitive deep network structure optimization. The algorithm analyzes the connection between different convolutional layer features and animation characteristics through animation creation style CNN feature visualization. It interactively uses the knowledge of animation creation psychology expression techniques to optimize the network structure. This paper proposes a scene animation network based on spectral difference perception style. By analyzing the characteristics and differences in the spectrum between realistic and anime domain images, the generator is guided to learn the mapping relationships better to fit the style distribution of anime domain images. This paper uses a fully convolutional structure; the network is more lightweight and supports image inputs of arbitrary size, which can keep the semantic system of the background unchanged while highly deforming the five facial features, moving toward the goal of human-scene fusion for the animation task.

Online signature verification using deep learning based aggregated convolutional feature representation

Handwritten signature for the identification and authentication of an individual has been widely used in the biometric systems. Due to the intra-class and inter-class variabilities, signature verification has become one of the most challenging problem in the biometric technology. Furthermore, the offline handwritten signature can be forged by the skilled persons due to its static nature. Therefore, in this paper a deep learning-based method using convolutional neural network (CNN) for online signature verification has been developed. Different values of the convolutional kernels such as 1×1, 3×3 and 5×5 are used to extract the discriminative features at multi-scales. The features of the initial and middle layers of the CNN are combined to create more powerful features. An up-sampling method with bilinear interpolation has been used to add the features of convolutional layers with different spatial dimensions. Both the addition and concatenation methods have been used to aggregate the convolutional features. A convolutional transpose method is applied to increase the depth of the convolutional layers while performing an addition operation on the layers with different depths. Finally, the concatenated features are passed to the fully connected layers for high-level feature extraction and classification. To evaluate the performance of the proposed method, an android application was developed where; a custom database of 985 online signatures collected from 197 users has been created. The problem of inadequate training data for online signature verification has been addressed through the data augmentation method. The experimental results show that the deep aggregated convolutional feature representation method achieves an accuracy of 99.32% on the custom developed online signature database.

Breast cancer histopathological images classification based on deep semantic features and gray level co-occurrence matrix.

Breast cancer is regarded as the leading killer of women today. The early diagnosis and treatment of breast cancer is the key to improving the survival rate of patients. A method of breast cancer histopathological images recognition based on deep semantic features and gray level co-occurrence matrix (GLCM) features is proposed in this paper. Taking the pre-trained DenseNet201 as the basic model, part of the convolutional layer features of the last dense block are extracted as the deep semantic features, which are then fused with the three-channel GLCM features, and the support vector machine (SVM) is used for classification. For the BreaKHis dataset, we explore the classification problems of magnification specific binary (MSB) classification and magnification independent binary (MIB) classification, and compared the performance with the seven baseline models of AlexNet, VGG16, ResNet50, GoogLeNet, DenseNet201, SqueezeNet and Inception-ResNet-V2. The experimental results show that the method proposed in this paper performs better than the pre-trained baseline models in MSB and MIB classification problems. The highest image-level recognition accuracy of 40×, 100×, 200×, 400× is 96.75%, 95.21%, 96.57%, and 93.15%, respectively. And the highest patient-level recognition accuracy of the four magnifications is 96.33%, 95.26%, 96.09%, and 92.99%, respectively. The image-level and patient-level recognition accuracy for MIB classification is 95.56% and 95.54%, respectively. In addition, the recognition accuracy of the method in this paper is comparable to some state-of-the-art methods.

Approximate Down-Sampling Strategy for Power-Constrained Intelligent Systems

In modern power constrained applications, as with most of those belonging to the Internet-of-Things world, custom hardware supports are ever more commonly adopted to deploy artificial intelligence algorithms. In these operating environments, limiting the power dissipation as much as possible is mandatory, even at the expense of reduced computational accuracy. In this paper we propose a novel prediction method to identify potential predominant features in convolutional layers followed by down-sampling layers, thus reducing the overall number of convolution calculations. This approximation down-sampling strategy has been exploited to design a custom hardware architecture for the inference of Convolutional Neural Network (CNN) models. The proposed approach has been applied to several benchmark CNN models and we achieved an overall energy saving of up to 70% with an accuracy loss lower than 3%, with respect to baseline designs. Performed experiments demonstrate that, when adopted to infer the Visual Geometry Group-16 (VGG16) network model, the proposed architecture implemented on a Xilinx Z-7045 chip and on the STM 28nm process technology dissipates only 680 and 21.9 mJ/frame, respectively. In both cases, the novel design overcomes several state-of-the-art competitors in terms of energy-accuracy drop product.

GCB-Net: Graph Convolutional Broad Network and Its Application in Emotion Recognition

In recent years, emotion recognition has become a research focus in the area of artificial intelligence. Due to its irregular structure, EEG data can be analyzed by applying graphical based algorithms or models much more efficiently. In this work, a Graph Convolutional Broad Network (GCB-net) was designed for exploring the deeper-level information of graph-structured data. It used the graph convolutional layer to extract features of graph-structured input and stacks multiple regular convolutional layers to extract relatively abstract features. The final concatenation utilized the broad concept, which preserves the outputs of all hierarchical layers, allowing the model to search features in broad spaces. To improve the performance of the proposed GCB-net, the broad learning system (BLS) was applied to enhance its features. For comparison, two individual experiments were conducted to examine the efficiency of the proposed GCB-net based on the SJTU emotion EEG dataset (SEED) and DREAMER dataset respectively. In SEED, compared with other state-of-art methods, the GCB-net could better promote the accuracy (reaching 94.24 percent) on the DE feature of the all-frequency band. In DREAMER dataset, GCB-net performed better than other models with the same setting. Furthermore, the GCB-net reached high accuracies of 86.99, 89.32 and 89.20 percent on dimensions of Valence, Arousal and Dominance respectively. The experimental results showed the robust classifying ability of the GCB-net and BLS in EEG emotion recognition.

Person Re-Identification by Context-Aware Part Attention and Multi-Head Collaborative Learning

Most existing works solve the video-based person re-identification (re-ID) problem by computing the representation of each frame independently and finally aggregate the frame-level features. However, these methods often suffer from the challenging factors in videos, such as serious occlusion, background clutter and pose variation. To address these issues, we propose a novel multi-level Context-aware Part Attention (CPA) model to learn discriminative and robust local part features. It is featured in two aspects: 1) the context-aware part attention module improves the robustness by capturing the global relationship among different body parts across different video frames, and 2) the attention module is further extended to multi-level attention mechanism which enhances the discriminability by simultaneously considering low- to high-level features in different convolutional layers. In addition, we propose a novel multi-head collaborative training scheme to improve the performance, which is collaboratively supervised by multiple heads with the same structure but different parameters. It contains two consistency regularization terms, which consider both multi-head and multi-frame consistency to achieve better results. The multi-level CPA model is designed for feature extraction, while the multi-head collaborative training scheme is designed for classifier supervision. They jointly improve our re-ID model from two complementary directions. Extensive experiments demonstrate that the proposed method achieves much better or at least comparable performance compared to the state-of-the-art on four video re-ID datasets.

Multimodal Cross-Layer Bilinear Pooling for RGBT Tracking

Hierarchical deep features can provide multilevel abstractions of target objects, which play an important role in target localization and classification. Determining how to effectively aggregate abstract information from different levels in RGB and thermal modalities is the key to exploiting their complementary advantages for robust RGBT tracking. However, existing RGBT tracking algorithms either focus on the semantic information of the last layer or aggregate hierarchical deep features from each modal using simple operations (e.g., summation and concatenation), which limit the capability of the multimodal tracker. To address these issues, in this paper, we propose a novel multimodal cross-layer bilinear pooling network for RGBT tracking. In our network, firstly, to boost the performance of the tracker, we use a channel attention mechanism to implement the adaptive calibration of feature channels for all convolutional layer features before realizing hierarchical feature fusion. Then, a bilinear pooling operation is performed on any two layers through the cross product, which is a second-order computation that effectively aggregates the deep semantic and shallow texture information of the target. Finally, a quality-aware fusion module is designed to aggregate the bilinear pooling features of different layer interactions between different modalities in an adaptive manner. The results of a large number of experiments on two public benchmark datasets demonstrate the effectiveness of our tracker compared with other state-of-the-art tracking methods.

Deep Learning Model for Wind Forecasting: Classification Analyses for Temporal Meteorological Data

This paper proposes a multiple CNN architecture with multiple input features, combined with multiple LSTM, along with densely connected convolutional layers, for temporal wind nature analyses. The designed architecture is called Multiple features, Multiple Densely Connected Convolutional Neural Network with Multiple LSTM Architecture, i.e. MCLT. A total of 58 features in the input layers of the MCLT are designed using wind speed and direction values. These empirical features are based on percentage difference, standard deviation, correlation coefficient, eigenvalues, and entropy, for efficiently describing the wind trend. Two successive LSTM layers are used after four densely connected convolutional layers of the MCLT. Moreover, LSTM has memory units that utilise learnt features from the current as well as previous outputs of the neurons, thereby enhancing the learning of patterns in the temporal wind dataset. Densely connected convolutional layer helps to learn features of other convolutional layers as well. The MCLT is used to predict dominant speed and direction classes in the future for the wind datasets of Stuttgart and Netherlands. The maximum and minimum overall accuracies for dominant speed prediction are 99.1% and 94.9%, (for Stuttgart) and 99.9% and 97.5% (for Netherlands) and for dominant direction prediction are 99.9% and 94.4% (for Stuttgart) and 99.6% and 96.4% (for Netherlands), respectively, using MCLT with 58 features. The MCLT, therefore, with multiple features at different levels, i.e. the input layers, the convolutional layers, and LSTM layers, shows promising results for the prediction of dominant speed and direction. Thus, this work is useful for proper wind utilisation and improving environmental planning. These analyses would also help in performing Computational Fluid Dynamics (CFD) simulations using wind speed and direction measured at a nearby meteorological station, for devising a new set of appropriate inflow boundary conditions.

Farm-Scale Crop Yield Prediction from Multi-Temporal Data Using Deep Hybrid Neural Networks

Farm-scale crop yield prediction is a natural development of sustainable agriculture, producing a rich amount of food without depleting and polluting environmental resources. Recent studies on crop yield production are limited to regional-scale predictions. The regional-scale crop yield predictions usually face challenges in capturing local yield variations based on farm management decisions and the condition of the field. For this research, we identified the need to create a large and reusable farm-scale crop yield production dataset, which could provide precise farm-scale ground-truth prediction targets. Therefore, we utilise multi-temporal data, such as Sentinel-2 satellite images, weather data, farm data, grain delivery data, and cadastre-specific data. We introduce a deep hybrid neural network model to train this multi-temporal data. This model combines the features of convolutional layers and recurrent neural networks to predict farm-scale crop yield production across Norway. The proposed model could efficiently make the target predictions with the mean absolute error of 76 kg per 1000 m2. In conclusion, the reusable farm-scale multi-temporal crop yield dataset and the proposed novel model could meet the actual requirements for the prediction targets in this paper, providing further valuable insights for the research community.

Learning to capture dependencies between global features of different convolution layers

NLNet has been considered as one milestone in the study of capturing long-range dependencies. Many recent studies modify the internal structure of NLNet directly and apply them to video object detection and semantic segmentation tasks. The dependencies between local and global features have been well developed, but the dependencies between global features of different convolution layers are rarely considered. Convolution is a local operation, so the global features of different convolution layers cannot be directly related, resulting in the loss of dependencies between global features. Given the vulnerability, this study designs a network that can efficiently capture the dependencies between the global features of different convolution layers, potentially further improving the accuracy. Furthermore, for the calculation of the dependency matrix, based on the Dot-product used in NLNet, we propose RELU-Dot-product, which can achieve higher accuracy. We evaluatethe proposed method on image classification and object detection tasks. The data sets involved are CIFAR10, CIFAR100, Tiny-imagenet, VOC2007, VOC2012 and MS COCO. Experiments show that our method can significantly improve network performance by introducing a few parameters.