Convolutional Neural Network-based Method Research Articles

A Convolutional Neural Network-Based Quantization Method for Block Compressed Sensing of Images.

Block compressed sensing (BCS) is a promising method for resource-constrained image/video coding applications. However, the quantization of BCS measurements has posed a challenge, leading to significant quantization errors and encoding redundancy. In this paper, we propose a quantization method for BCS measurements using convolutional neural networks (CNN). The quantization process maps measurements to quantized data that follow a uniform distribution based on the measurements' distribution, which aims to maximize the amount of information carried by the quantized data. The dequantization process restores the quantized data to data that conform to the measurements' distribution. The restored data are then modified by the correlation information of the measurements drawn from the quantized data, with the goal of minimizing the quantization errors. The proposed method uses CNNs to construct quantization and dequantization processes, and the networks are trained jointly. The distribution parameters of each block are used as side information, which is quantized with 1 bit by the same method. Extensive experiments on four public datasets showed that, compared with uniform quantization and entropy coding, the proposed method can improve the PSNR by an average of 0.48 dB without using entropy coding when the compression bit rate is 0.1 bpp.

A fiber recognition framework based on multi-head attention mechanism

Convolutional neural networks have been extensively studied in textile fiber recognition. However, the down-sampling performed by the convolutional and pooling layers on the extracted features result in a significant loss of fine-grained fiber features. To address this issue, a multi-head attention framework for fiber recognition has been proposed. First, textile surface images are captured using optical magnifiers and smart devices such as smartphones. Second, fiber features and label embeddings are learned separately by multi-head self-attention. Finally, the label embeddings are used to query the presence of fiber types, and pool type-related features in the multi-head cross-attention module to classify fibers. The experimental results demonstrate that the proposed method performs exceptionally well on the textile surface image dataset, with a mean average precision accuracy improvement of 6% compared with the convolutional neural network-based fiber recognition method Cu-Net, and a 5% improvement compared with the fiber recognition method FiberCT combining convolutional neural networks and attention mechanisms. It is worth mentioning that the fiber images captured at 200× magnification in the collected dataset are most favorable for models to recognize fiber. The proposed method achieves a mean average precision fiber recognition accuracy of 80.2% on images at 200× magnification.

Efficient Target Classification Based on Vehicle Volume Estimation in High-Resolution Radar Systems

In this paper, we propose a method for efficient target classification based on the spatial features of the point cloud generated by using a high-resolution radar sensor. The frequency-modulated continuous wave radar sensor can estimate the distance and velocity of a target. In addition, the azimuth and elevation angle of the target can be estimated by using a multiple-input and multiple-output antenna system. Using the estimated distance, velocity, and angle, the 3D point cloud of target can be generated. From the generated point cloud, we extract the point cloud for each individual target using the density-based spatial clustering of application with noise method and a camera mounted on the radar sensor. Then, we define the convex hull boundaries that enclose these point clouds in both 3D and 2D spaces obtained by orthogonally projecting onto the xy, yz, and zx planes. Using the vertices of convex hull, we calculate the volume of the targets and the areas in 2D spaces. Several feature points, including the calculated spatial information, are numerized and configured into feature vectors. We design an uncomplicated deep neural network classifier based on minimal input information to achieve fast and efficient classification performance. As a result, the proposed method achieved an average accuracy of 97.1%, and the time required for training was reduced compared to the method using only point cloud data and the convolutional neural network-based method.

An efficient approach for automatic crack detection using deep learning

PurposeAutomation of detecting cracked surfaces on buildings or in any industrially manufactured products is emerging nowadays. Detection of the cracked surface is a challenging task for inspectors. Image-based automatic inspection of cracks can be very effective when compared to human eye inspection. With the advancement in deep learning techniques, by utilizing these methods the authors can create automation of work in a particular sector of various industries.Design/methodology/approachIn this study, an upgraded convolutional neural network-based crack detection method has been proposed. The dataset consists of 3,886 images which include cracked and non-cracked images. Further, these data have been split into training and validation data. To inspect the cracks more accurately, data augmentation was performed on the dataset, and regularization techniques have been utilized to reduce the overfitting problems. In this work, VGG19, Xception and Inception V3, along with Resnet50 V2 CNN architectures to train the data.FindingsA comparison between the trained models has been performed and from the obtained results, Xception performs better than other algorithms with 99.54% test accuracy. The results show detecting cracked regions and firm non-cracked regions is very efficient by the Xception algorithm.Originality/valueThe proposed method can be way better back to an automatic inspection of cracks in buildings with different design patterns such as decorated historical monuments.

A deep inverse convolutional neural network-based semantic classification method for land cover remote sensing images

The imbalance of land cover categories is a common problem. Some categories appear less frequently in the image, while others may occupy the vast majority of the proportion. This imbalance can lead the classifier to tend to predict categories with higher frequency of occurrence, while the recognition effect on minority categories is poor. In view of the difficulty of land cover remote sensing image multi-target semantic classification, a semantic classification method of land cover remote sensing image based on depth deconvolution neural network is proposed. In this method, the land cover remote sensing image semantic segmentation algorithm based on depth deconvolution neural network is used to segment the land cover remote sensing image with multi-target semantic segmentation; Four semantic features of color, texture, shape and size in land cover remote sensing image are extracted by using the semantic feature extraction method of remote sensing image based on improved sequential clustering algorithm; The classification and recognition method of remote sensing image semantic features based on random forest algorithm is adopted to classify and identify four semantic feature types of land cover remote sensing image, and realize the semantic classification of land cover remote sensing image. The experimental results show that after this method classifies the multi-target semantic types of land cover remote sensing images, the average values of Dice similarity coefficient and Hausdorff distance are 0.9877 and 0.9911 respectively, which can accurately classify the multi-target semantic types of land cover remote sensing images.

A Phenotypic Extraction and Deep Learning-Based Method for Grading the Seedling Quality of Maize in a Cold Region

Background: Low-temperature stress significantly restricts maize germination, seedling growth and development, and yield formation. However, traditional methods of evaluating maize seedling quality are inefficient. This study established a method of grading maize seedling quality based on phenotypic extraction and deep learning. Methods: A pot experiment was conducted using different low-temperature combinations and treatment durations at six different stages between the sowing and seedling phases. Changes in 27 seedling quality indices, including plant morphology and photosynthetic performance, were investigated 35 d after sowing and seedling quality grades were classified based on maize yield at maturity. The 27 quality indices were extracted, and a total of 3623 sample datasets were obtained and grouped into training and test sets in a 3:1 ratio. A convolutional neural network-based grading method was constructed using a deep learning model. Results: The model achieved an average precision of 98.575%, with a recall and F1-Score of 98.7% and 98.625%, respectively. Compared with the traditional partial least squares and back propagation neural network, the model improved recognition accuracy by 8.1% and 4.19%, respectively. Conclusions: This study provided an accurate grading of maize seedling quality as a reference basis for the standardized production management of maize in cold regions.

Prediction of CRISPR/Cas9 off-target activity using multi-scale convolutional neural network

Clustered regularly interspaced short palindromic repeat/CRISPR-associated protein 9 (CRISPR/Cas9) is a new generation of gene editing technology, which relies on single guide RNA to identify specific gene sites and guide Cas9 nuclease to edit specific location in the genome. However, the off-target effect of this technology hampers its development. In recent years, several deep learning models have been developed for predicting the CRISPR/Cas9 off-target activity, which contributes to more efficient and safe gene editing and gene therapy. However, the prediction accuracy remains to be improved. In this paper, we proposed a multi-scale convolutional neural network-based method, designated as CnnCRISPR, for CRISPR/Cas9 off-target prediction. First, we used one-hot encoding method to encode the sgRNA-DNA sequence pair, followed by a bitwise or operation on the two binary matrices. Second, the encoded sequence was fed into the Inception-based network for training and evaluating. Third, the well-trained model was applied to evaluate the off-target situation of the sgRNA-DNA sequence pair. Experiments on public datasets showed CnnCRISPR outperforms existing deep learning-based methods, which provides an effective and feasible method for addressing the off-target problems.

GTMFuse: Group-attention transformer-driven multiscale dense feature-enhanced network for infrared and visible image fusion

Infrared and visible images captured by different devices can be seamlessly integrated into a single composite image through the application of image fusion techniques. However, many existing convolutional neural network-based methods for infrared and visible image fusion have exhibited limited capability for effectively amalgamating information from the source images. Consequently, we propose a group-attention transformer into the multiscale feature enhanced network for infrared and visible image fusion, which we abbreviate as GTMFuse. Specifically, GTMFuse employs multiscale dual-channel encoders to independently process the source image and extract multiscale features. Among the encoders, the group-attention transformer module is utilized to facilitate more comprehensive long-range feature dependency modeling at each scale. This innovative module seamlessly combines a fixed-direction stripe attention mechanism with channel attention and window attention, enabling comprehensive global long-range information capture and interaction with feature information across the source images. The multiscale features obtained from the group-attention transformer module are integrated into the fused image through a meticulously designed dense fusion block. Furthermore, this study introduces a novel dataset named HBUT-IV, encompassing surveillance images captured from multiple viewpoints. The HBUT-IV dataset serves as a valuable benchmark for assessing the efficacy of fusion methods. Extensive experiments are conducted on four datasets employing nine comparative methods, revealing the superior performance of the GTMFuse approach. The implementation code is accessible at https://github.com/XingLongH/GTMFuse.

Fault diagnosis and identification of rotating machinery based on one-dimensional convolutional neural network

The paper focuses on two kinds of rotating machinery, miniature table drilling machine and automobile engine, as the research object. Traditional machine learning has the need for manual feature extraction, and is very dependent on expert diagnostic experience and expertise, but also has the disadvantages of low accuracy, low timeliness, low efficiency, etc. For the traditional rotating machinery fault diagnosis method is more based on the traditional machine learning model, this paper puts forward a one-dimensional convolutional neural network-based fault identification method. According to the characteristics of the miniature table drilling machine and the automobile engine which are not detachable, the corresponding faults are set up respectively, Vibration signals of the attitude sensor are obtained by using the signal collector, and the collected data are preprocessed, then the CNN model is built for fault identification, and the network structure is constantly optimized to obtain the optimal network model with high accuracy (up to 100 %) and robustness. The results show that the one-dimensional convolutional neural network model improves the fault recognition accuracy and reduces the cost compared with the traditional machine learning SVM model when the original signal is used as the input signal.

AiPE: A Novel Transformer-Based Pose Estimation Method

AiPE: A Novel Transformer-Based Pose Estimation Method

Research on Convolutional Neural Network-Based Compression Methods for Multispectral Images

Considering the traits of multispectral images, which include numerous bands, high spatial and spectral redundancy, and a large data volume, there is a proposed investigation on compression methods based on convolutional neural networks to reduce the storage space consumed by an individual image and enhance compression effectiveness. This paper first introduces the development of image compression algorithms and deep learning in recent years. Based on these two structures, a framework for lossy compression of multispectral images utilizing an end-to-end convolutional neural network is proposed. A self-encoding structure is used to process three-dimensional hyperspectral images, extracting local spectral features and fusing spectral information using large convolutional kernels. Residual layers are employed to preserve spectral information. Rate-distortion optimization is performed to jointly optimize image distortion and compression bitrate. Finally, a comparison with the traditional JPEG method is conducted experiments to assess the efficacy of the proposed algorithm. The MS-SSIM is improved by nearly 0.08, and the compressed images exhibit no noticeable distortion.

An Improved Convolution Neural Network-Based Fast Estimation Method for Construction Project Cost

At present, most network models do not fully consider the prior information on construction project cost, and do not do much research to find out more information or effectively utilize the information the find on construction project cost; conversely, some previous networks mainly consider the depth of the network for the reconstruction results. In addition, there is no detailed discussion on the selection of loss functions for various algorithms, and some common error loss functions are directly used. In view of these shortcomings, a fast estimation model of construction cost based on an improved convolutional neural network is proposed. The construction project is decomposed according to the project characteristics. The project characteristic factors that have a greater impact on the project cost and the main project quantity are selected and analyzed. Then, the convolutional neural network principle is used to analyze the project characteristic factors, the project cost and main project quantities. We simulate and compare the established convolutional neural network models, verify the usability of the model, and finally get a model with high reliability. The establishment of this model provides a new method for the estimation of project cost and main economic indicators, which can meet the actual work requirements to a certain extent.

An enhanced Swin Transformer for soccer player reidentification

The re-identification (ReID) of objects in images is a widely studied topic in computer vision, with significant relevance to various applications. The ReID of players in broadcast videos of team sports is the focus of this study. We specifically focus on identifying the same player in images taken at any given moment during a game from various camera angles. This work varies from other person ReID apps since the same team wears very similar clothes, there are few samples for each identification, and image resolutions are low. One of the hardest parts of object ReID is robust feature representation extraction. Despite the great success of current convolutional neural network-based (CNN) methods, most studies only consider learning representations from images, neglecting long-range dependency. Transformer-based model studies are increasing and yielding encouraging results. Transformers still have trouble extracting features from small objects and visual cues. To address these issues, we enhanced the Swin Transformer with the levering of CNNs. We created a regional feature extraction Swin Transformer (RFES) backbone to increase local feature extraction and small-scale object feature extraction. We also use three loss functions to handle imbalanced data and highlight challenging situations. Re-ranking with k-reciprocal encoding was used in this study's retrieval phase, and its assessment findings were provided. Finally, we conducted experiments on the Market-1501 and SoccerNet-v3 ReID datasets. Experimental results show that the proposed re-ID method reaches rank-1 accuracy of 96.2% with mAP: 89.1 and rank-1 accuracy of 84.1% with mAP: 86.7 on the Market-1501 and SoccerNet-v3 datasets, respectively, outperforming the state-of-the-art approaches.

Artificial Intelligence and Malicious Code Detection

Malicious software has become a critical cybersecurity issue due to the increasing number of threats it poses to devices and internet environments. Traditional static scanning techniques and behavior-based malware detection methods have limitations in meeting the new requirements in information security due to high false positives and false negatives. In this work, we propose a CNN convolutional neural network-based method for detecting malicious code. Practical operations were conducted using the Cuckoo sandbox system, and Python programs were utilized to preprocess analytical reports. This article presents the construction of a deep learning training model for CNN designed to identify malicious code. The model is compared with machine learning and general antivirus tools for comprehensive evaluation. Experimental verification shows that our proposed method exhibits greater advantages in comparison and achieved excellent detection results with higher feasibility. The research significance of this work is highlighted as malicious software has become a core issue in cybersecurity. The method proposed in this article provides powerful support for addressing new needs in information security. This paper emphasizes the importance of utilizing CNN-based methods in detecting malware, which can better address the limitations of traditional detection techniques. Overall, this work provides an effective solution to detect malicious code and addresses a critical cybersecurity issue.

A convolutional neural network-based method for the generation of super-resolution 3D models from clinical CT images

Background and objectiveThe accurate evaluation of bone mechanical properties is essential for predicting fracture risk based on clinical computed tomography (CT) images. However, blurring and noise in clinical CT images can compromise the accuracy of these predictions, leading to incorrect diagnoses. Although previous studies have explored enhancing trabecular bone CT images to super-resolution (SR), none of these studies have examined the possibility of using clinical CT images from different instruments, typically of lower resolution, as a basis for analysis. Additionally, previous studies rely on 2D SR images, which may not be sufficient for accurate mechanical property evaluation, due to the complex nature of the 3D trabecular bone structures. The objective of this study was to address these limitations. MethodsA workflow was developed that utilizes convolutional neural networks to generate SR 3D models across different clinical CT instruments. The morphological and finite-element-derived mechanical properties of these SR models were compared with ground truth models obtained from micro-CT scans. ResultsA significant improvement in analysis accuracy was demonstrated, where the new SR models increased the accuracy by up to 700 % compared with the low-resolution data, i.e. clinical CT images. Additionally, we found that the mixture of different CT image datasets may improve the SR model performance. ConclusionsSR images, generated by convolutional neural networks, outperformed clinical CT images in the determination of morphological and mechanical properties. The developed workflow could be implemented for fracture risk prediction, potentially leading to improved diagnoses and subsequent clinical decision making.

Printer source identification of quick response codes using residual attention network and smartphones

Quick response codes are widely utilized in commodity anti-counterfeiting, and their traceability via smartphones is an effective authentication method. However, quick response codes are susceptible to illegal duplication, and the images captured by smartphones are easily affected by changes in light and environmental noise, leading to unsatisfactory verification results. To address these issues, this paper introduces a novel approach by combining the squeeze-excitation attention module with bottleneck residual block. It presents a squeeze-excitation bottleneck residual network for printer source identification of quick response codes. The squeeze-excitation attention module pays more attention to the features that represent the printer attributes, reduces the interference of useless information, and has low computational consumption; while the bottleneck residual block has the advantages of few parameters, strong feature extraction capability, and good expandability. Thus, the performance can be improved effectively only with a small increasing in parameters. The experimental results verify that the proposed method achieves an accuracy of 98.77% under smartphone capture conditions, it outperforms other convolutional neural network-based methods in terms of identification accuracy. The deep learning model proposed in this paper can be generalized and applied to the printer source identification of paper content in the civil, criminal investigation and judicial fields.

Cross-Scale Fusion Transformer for Histopathological Image Classification.

Histopathological images provide the medical evidences to help the disease diagnosis. However, pathologists are not always available or are overloaded by work. Moreover, the variations of pathological images with respect to different organs, cell sizes and magnification factors lead to the difficulty of developing a general method to solve the histopathological image classification problems. To address these issues, we propose a novel cross-scale fusion (CSF) transformer which consists of the multiple field-of-view patch embedding module, the transformer encoders and the cross-fusion modules. Based on the proposed modules, the CSF transformer can effectively integrate patch embeddings of different field-of-views to learn cross-scale contextual correlations, which represent tissues and cells of different sizes and magnification factors, with less memory usage and computation compared with the state-of-the-art transformers. To verify the generalization ability of the CSF transformer, experiments are performed on four public datasets of different organs and magnification factors. The CSF transformer outperforms the state-of-the-art task specific methods, convolutional neural network-based methods and transformer-based methods. The source code will be available in our GitHub https://github.com/nchucvml/CSFT.

A Deep Convolutional Neural Network-Based Method for Self-Piercing Rivet Joint Defect Detection

Abstract The self-pierce riveting process for alloy materials has a wide range of applications in the automotive manufacturing industry. This will not only affect the operation performance but also cause accidents in severe cases when there are defects in the riveted parts. A deep learning detection model is proposed that integrates atrous convolution and dynamic convolution to identify defects of self-piercing riveting parts efficiently to overcome the problem in quality inspection after the body self-piercing riveting process. First, a backbone network for extracting riveting defect features is constructed based on the ResNet network. Second, the center area of each riveting defect is located preferentially by the center point detection algorithm. Finally, the bounding box of riveting defects is regressed to achieve defect detection based on this central region. Among them, atrous convolution is used in the external network to increase the receptive field of the model, which combined with an active convolution so that a dynamic atrous convolution module is designed. This module is used to enhance the correlation between feature points of individual pixel in the image, which helps to identify defects with incomplete image edges and suppress background interference. Ablation experiments show that the proposed method achieves the highest accuracy of 96.3%, which is 3.9% higher than the original method. It is found that the proposed method is less affected by the background interference from the qualitative comparison. Moreover, it can also effectively identify the riveting defects on the surface of each area.

Content-based image retrieval for medical diagnosis using fuzzy clustering and deep learning

Brain tumors are one of the most threatening causes of death worldwide. Neuroradiologists widely use Magnetic Resonance Imaging as a golden standard non-invasive imaging method for diagnosing Glioma grades. Brain tumor diagnosis based on radiology images is tedious and highly subjective to intra and inter-observer variability; it adversely affects therapeutic planning. Better medical treatment can be provided relating to similar past cases if relevant images are retrieved from an extensive medical image database. Content-Based Image Retrieval systems are potent image tools for dealing with such massive datasets. In a CBIR system, accurate classification and retrieval of similar pathological images can be effectively automated by utilising Convolutional Neural Network-based feature extraction methods. This paper presents a Content-Based Medical Image Retrieval pipeline on a medical domain using the CNN model for feature extraction and the clustering method used to index the feature map database. The proposed system applies a Multi-level Gain-based feature selection to reduce the dimensionality of the feature vectors obtained from the pre-trained CNN models. The experiment follows five-fold cross-validation on BraTS 2018 and 2020 datasets. The proposed system achieves state-of-the-art MRI brain image retrieval performance with a mean Average Precision and Precision@10 of 98.15% and 97.62%, respectively. The results prove the proposed method’s effectiveness and validate its clinical application.

Anchor-free deep convolutional neural network for tracking and counting cotton seedlings and flowers

Accurate counting of plants and their organs in natural environments is essential for breeders and growers. For breeders, counting plants during the seedling stage aids in selecting genotypes with superior emergence rates, while for growers, it informs decisions about potential replanting. Meanwhile, counting specific plant organs, such as flowers, forecasts yields for different genotypes, offering insights into production levels. The overall goal of this study was to investigate a deep convolutional neural network-based tracking method, CenterTrack, for cotton seedling and flower counting from video frames. The network is extended from a customized CenterNet, which is an anchor-free object detector. CenterTrack predicts the detections of the current frame and displacements of detections between the previous frame and the current frame, which are used to associate the same object in consecutive frames. The modified CenterNet detector achieved high accuracy on both seedling and flower datasets with an overall AP50 of 0.962. The video tracking hyperparameters were optimized for each dataset using orthogonal tests. Experimental results showed that seedling and flower counts with optimized hyperparameters highly correlated with those of manual counts (R2 = 0.98 andR2 = 0.95) and the mean relative errors of 75 cotton seedling testing videos and 50 flower testing videos were 5.5 % and 10.8 %, respectively. An average counting speed of 20.4 frames per second was achieved with an input resolution of 1920 × 1080 pixels for both seedling and flower videos. The anchor-free deep convolution neural network-based tracking method provides automatic tracking and counting in video frames, which will significantly benefit plant breeding and crop management.