Convolutional Neural Network Structure Research Articles

A 3D-2DCNN-CA approach for enhanced classification of hickory tree species using UAV-based hyperspectral imaging

Hickory trees possess substantial economic, nutritional, and ecological value, and they play a pivotal role in both human societies and natural ecosystems. To maximize benefits, many bases have adopted a composite planting strategy. Effective and timely monitoring of hickory forest species information is crucial for precise management and conservation. In this study, a low-altitude unmanned aerial vehicle (UAV) was employed to capture RGB and hyperspectral data from the canopy of two hickory bases. A hybrid convolutional neural network structure was then utilized to classify different tree species and congeneric hickory species. By introducing a channel attention module to refine the features of hyperspectral images, classification stability was enhanced. The experimental results demonstrate that hyperspectral images yield superior classification performance in hickory species identification compared to RGB images, particularly in classifying highly homogeneous tree species. The 3D-2DCNN-CA proposed in this paper demonstrated the best performance in classifying hickory species compared to other classification methods. In the classification of different tree species, the accuracy for a single hickory reached 99.38%, and in the classification of the same hickory species, it achieved an accuracy of 93.97%. Furthermore, this method also achieved significant classification results at the single-tree scale. These results indicate that the method can realize fine-scale monitoring of hickory forests and provide substantial support for forest land management and expert guidance on planting distribution.

A novel electronic nose classification prediction method based on TETCN

An efficient algorithm model is proposed to achieve good performance for gas detection based on electronic nose (E-nose) system. Transformer Encoder (TE) has been widely used in natural language processing and shown excellent performance in handling sequence data, which could help the model to learn the contextual information of E-nose data. Temporal Convolutional Network (TCN) is a novel convolutional neural network structure with a receptive field variable in length and the ability to capture long-term dependence, which is suitable for processing time series data. This work proposes a gas classification method based on TETCN which is composed of TE and TCN for processing the data of E-nose. Since hyperparameters have a significant impact on model performance, the Bayesian parameter optimization algorithm is used for TETCN. The data is weighted by TE, and then TCN can easily extract important features, resulting in a satisfactory classification accuracy of 99.8%. The experimental results indicate TETCN has better performance than the conventional methods such as convolutional neural network (CNN), long-short term memory (LSTM), and gated recurrent unit (GRU). Furthermore, network ablation has been implemented to demonstrate the necessity of combining TE and TCN. Finally, the feasibility of rapid detection of the proposed algorithm is discussed.

AIR-CNN: a lightweight automatic image rectification CNN used for barrel distortion

Barrel distortions often exist in images captured by wide-angle lenses, and the presence of barrel distortions reduces the label-making accuracy of algorithms and the precision rate of final target detection and semantic recognition. To reduce the interference of barrel distortions on imaging, we have proposed a lightweight image rectification network automatic image rectification CNN (AIR-CNN) for barrel distortion. The network is based on a parameter sharing (PS) convolutional neural network structure, which is trained on the distorted image dataset to predict the pixel displacement field between the distorted image and the rectified image, and finally restores the rectified image based on the predicted pixel displacement field. The experimental results show that the AIR-CNN can greatly reduce the number of network parameters through the PS mechanism and implicitly learns the texture features by asymmetric convolution kernels to obtain higher rectification accuracy at a lower computational cost, and automatically obtain the distortion parameters of the camera without special calibration methods. The AIR-CNN outperforms previous image rectification methods in both intuitive and quantitative comparisons (EPE, PSNR, NRMSE, SSIM).

Research on Image Detection Algorithm of Rail Traffic Congestion Degree Based on Convolutional Neural Networks

With the sustainable development of the social economy and the continuous maturity of science and technology, urban rail transit has developed rapidly. It solved the problems of urban road load and people’s travel and brought about the problem of rail transit passenger congestion. The image detection algorithm for rail transit congestion is established based on the convolutional neural networks (CNN) structure to realize intelligent video image monitoring. The CNN structure is optimized through the backpropagation (BP) algorithm so that the model can detect and analyze the riding environment through the monitoring camera and extract the relevant motion characteristics of passengers from the image. Furthermore, the crowding situation of the riding environment is analyzed to warn the rail transit operators. In practical application, the detection accuracy of the algorithm reached 91.73%, and the image processing speed met the second-level processing. In the performance test, the proposed algorithm had the lowest mean absolute error (MAE) and mean square error (MSE). In Part B, the MAE and MSE values of the model were 16.3 and 24.9, respectively. The error values were small, so the performance was excellent. The purpose of this study is to reduce the possibility of abnormal crowd accidents at stations and provide new ideas for intelligent management of rail transit.

Vibration monitoring of structures with indirect load identification and Kalman update

The German Research Foundation (DFG) has launched in September 2022 the priority program SPP 2388 100+ to develop new methods for digital representation, SHM and lifetime management of complex structures, due to the continuously increasing amount of old infrastructure buildings. The present contribution is prepared within the LEMOTRA project as a part of SPP 2388 100+. Among various SHM methods, the approach based on the Kalman update for data assimilation between model and measurement is applied and further developed to create a kind of functional digital twin for SHM. For this purpose, a sound numerical model and a measurement system with a continuous data flow are introduced to provide online predictions of the state and response parameters of the structure. System changes resulting from damage or aging processes can be detected and localized, provided the measurement and the model prediction deal with the same cause. Thus, the load identification is a necessary prerequisite for reliable data assimilation techniques. A two-step update procedure is proposed and applied in this context. At first, one part of the measurement system is used for the load identification. A cluster structure of convolutional neural networks (CNNs) was developed, trained and calibrated to extract load characteristics such as load magnitudes, load velocities or the number of vehicles on the bridge from multiple acceleration signals. This information is then used to reconstruct the actual load. In the second step, a different set of sensors is used for the data assimilation. In contrast to the first set of measurement locations, the measurement data from these sensors should be sensitive to potential system changes or damage. Here, the identified load is used as input for the model predictions which are then compared to the measurement data. A combination of different ensemble based Kalman filters (KF) provides a sequential update of the state parameters (e.g. displacement, velocity, acceleration) and the model parameters (e.g. stiffness, mass, damping). The cluster CNN approach is tested numerically, and the data assimilation technique is tested on a laboratory structure.

STD-YOLOv8: A lightweight small target detection algorithm for UAV perspectives

<p>When recognizing targets by unmanned aerial vehicles (UAVs), problems such as small size, dense dispersion, and complex background are likely to occur, resulting in low recognition rates. In order to solve the above problems, this work proposed a lightweight small target detection algorithm based on the YOLOv8n: STD-YOLOv8 algorithm. First, the regression problem of small targets in different training periods was optimized, the penalty term in the original loss was improved, and a new LIoU loss function was proposed, so that the size of the penalty term could be dynamically adjusted before and after training, thus improving the performance of the algorithm. Second, in order to better adapt to the small target scale and enhance the ability of small target feature acquisition, the SPD-Conv module was integrated in the backbone network, replacing the original stepwise convolutional layer and pooling layer, so as to solve the problems of loss of fine-grained information and low efficiency of feature representation existing in the current convolutional neural network (CNN) structure. In the neck part, nearest-neighbor upsampling was replaced by the feature reassembly assembly of features operator CARAFE (content-aware reassembly of features), which enabled the model to aggregate contextual information in a larger perceptual field and enhanced the feature representation in the neck. Finally, validation experiments were conducted by comparing different algorithms under the same VisDrone-2021 dataset. The results of the ablation experiments show that the algorithms proposed in this thesis have improved the recall (R), mAP50, and mAP95 by 4.7, 5.8 and 5.7%, respectively, compared with YOLOv8n. The results of the model generalization experiments on the TinyPerson dataset show that the algorithm in this paper has superior small target detection performance with only 1.2 M model parameters (1 M = 10<sup>6</sup>).</p>

DEA-Net: Single Image Dehazing Based on Detail-Enhanced Convolution and Content-Guided Attention.

Single image dehazing is a challenging ill-posed problem which estimates latent haze-free images from observed hazy images. Some existing deep learning based methods are devoted to improving the model performance via increasing the depth or width of convolution. The learning ability of Convolutional Neural Network (CNN) structure is still under-explored. In this paper, a Detail-Enhanced Attention Block (DEAB) consisting of Detail-Enhanced Convolution (DEConv) and Content-Guided Attention (CGA) is proposed to boost the feature learning for improving the dehazing performance. Specifically, the DEConv contains difference convolutions which can integrate prior information to complement the vanilla one and enhance the representation capacity. Then by using the re-parameterization technique, DEConv is equivalently converted into a vanilla convolution to reduce parameters and computational cost. By assigning the unique Spatial Importance Map (SIM) to every channel, CGA can attend more useful information encoded in features. In addition, a CGA-based mixup fusion scheme is presented to effectively fuse the features and aid the gradient flow. By combining above mentioned components, we propose our Detail-Enhanced Attention Network (DEA-Net) for recovering high-quality haze-free images. Extensive experimental results demonstrate the effectiveness of our DEA-Net, outperforming the state-of-the-art (SOTA) methods by boosting the PSNR index over 41 dB with only 3.653 M parameters. (The source code of our DEA-Net is available at https://github.com/cecret3350/DEA-Net.).

A framework of evolutionary optimized convolutional neural network for classification of shang and chow dynasties bronze decorative patterns.

As a UNESCO World Cultural Heritage, the aesthetic value of bronze artifacts from the Shang and Chow Dynasties has had a profound influence on Chinese traditional culture and art. To facilitate the digital preservation and protection of these Shang and Chow bronze artifacts (SCB), it becomes imperative to categorize their decorative patterns. Therefore, a SCB pattern classification method of differential evolution called Shang and Chow Bronze Convolutional Neural Network (SCB-CNN) is proposed. Firstly, the original bronze decorative patterns of Shang and Chow dynasties are collected, and the samples are expanded through image augmentation technology to form a training dataset. Secondly, based on the classical convolutional neural network structure, the recognition and classification of bronze patterns are implemented by adjusting the network parameters. Then, the initial parameters of the convolutional neural network are optimized by differential evolution algorithm, and the optimized SCB-CNN is simulated. Finally, comparative experiments were conducted between the optimized SCB-CNN, the unoptimized model, VGG-Net, and GoogleNet. The experimental results indicate that the optimized SCB-CNN significantly reduces training time while maintaining fast prediction speed, convergence speed, and high accuracy. This study provides new insights for the inheritance and innovation research of SCB patterns.

Roadmap for providing and leveraging annotated data by cytologists in the PDAC domain as open data: support for AI-based pathology image analysis development and data utilization strategies.

Pancreatic cancer is one of the most lethal cancers worldwide, with a 5-year survival rate of less than 5%, the lowest of all cancer types. Pancreatic ductal adenocarcinoma (PDAC) is the most common and aggressive pancreatic cancer and has been classified as a health emergency in the past few decades. The histopathological diagnosis and prognosis evaluation of PDAC is time-consuming, laborious, and challenging in current clinical practice conditions. Pathological artificial intelligence (AI) research has been actively conducted lately. However, accessing medical data is challenging; the amount of open pathology data is small, and the absence of open-annotation data drawn by medical staff makes it difficult to conduct pathology AI research. Here, we provide easily accessible high-quality annotation data to address the abovementioned obstacles. Data evaluation is performed by supervised learning using a deep convolutional neural network structure to segment 11 annotated PDAC histopathological whole slide images (WSIs) drawn by medical staff directly from an open WSI dataset. We visualized the segmentation results of the histopathological images with a Dice score of 73% on the WSIs, including PDAC areas, thus identifying areas important for PDAC diagnosis and demonstrating high data quality. Additionally, pathologists assisted by AI can significantly increase their work efficiency. The pathological AI guidelines we propose are effective in developing histopathological AI for PDAC and are significant in the clinical field.

CFNet: Conditional filter learning with dynamic noise estimation for real image denoising

A mainstream type of the state of the arts (SOTAs) based on convolutional neural network (CNN) for real image denoising contains two sub-problems, i.e., noise estimation and non-blind denoising. This paper considers real noise approximated by heteroscedastic Gaussian/Poisson–Gaussian distributions with in-camera signal processing pipelines. The related works always exploit the estimated noise prior via channel-wise concatenation followed by a convolutional layer with spatially sharing kernels. Due to the variable modes of noise strength and frequency details of all feature positions, this design cannot adaptively tune the corresponding denoising patterns. To address this problem, we propose a novel conditional filter in which the optimal kernels for different feature positions can be adaptively inferred by local features from the image and the noise map. Also, we bring the thought that alternatively performs noise estimation and non-blind denoising into CNN structure, which continuously updates noise prior to guide the iterative feature denoising. In addition, according to the property of heteroscedastic Gaussian distribution, a novel affine transform block is designed to predict the stationary noise component and the signal-dependent noise component. Compared with SOTAs, extensive experiments are conducted on five synthetic datasets and four real datasets, which shows the improvement of the proposed CFNet. The code and models are available via https://github.com/WenhaoYao/CFNet/.

RFTNet: Region–Attention Fusion Network Combined with Dual-Branch Vision Transformer for Multimodal Brain Tumor Image Segmentation

Brain tumor image segmentation plays a significant auxiliary role in clinical diagnosis. Recently, deep learning has been introduced into multimodal segmentation tasks, which construct various Convolutional Neural Network (CNN) structures to achieve excellent performance. However, most CNN-based segmentation methods have poor capability for global feature extraction. Transformer is good at modeling long-distance dependencies, but it can cause local information loss and usually has a high computational complexity. In addition, it is difficult to fully exploit the brain tumor features of different modalities. To address these issues, in this paper, we propose a region–attention fusion (RAF) network that combines a dual-branch vision Transformer (DVT), called RFTNet. In RFTNet, the DVT is exploited to capture the delicate local information and global semantics separately by two branches. Meanwhile, a novel RAF is employed to effectively fuse the images of the different modalities. Finally, we design a new hybrid loss function, called region-mixed loss function (RML) to calculate the importance of each pixel and solve the problem of class imbalance. The experiments on BrasTS2018 and BraTS2020 datasets show that our method obtains a higher segmentation accuracy than other models. Furthermore, ablation experiments prove the effectiveness of each key component in RFTNet.

Sparse-firing regularization methods for spiking neural networks with time-to-first-spike coding

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first-spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike-timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.

Construction of a dynamic network for retinal vessel segmentation based on computer vision

This paper is focused on the field of computer vision in order to investigate the presentation properties of retinal blood vessels. Combining the structure of convolutional neural networks, activation functions, and common metrics in semantic segmentation, a dynamic network model for retinal vessel segmentation based on computer vision is constructed. The purpose of this paper is to discuss the results of retinal vascular segmentation based on computer vision. The image connection and alignment pattern selection process is also established to match retinal vessel images by computer vision. The performance of the dynamic network constructed here and the results of retinal vessel segmentation were then analyzed in three publicly available datasets, DRIVE (digital retinal images for vessel extraction), CHASE_DB1, and STARE (structured snalysis of the retinal. The ROC (retinopathy online challenge) curves on all three datasets exceeded 0.9, showing high performance, and the area under the PR curve exceeded 0.88. The accuracy of the results for retinal vessel segmentation was around 96%. Based on the semantic segmentation direction in the field of computer vision in this study, the dynamic network for retinal vessel segmentation can be well constructed.

Mask refinement network for tooth segmentation on panoramic radiographs.

Instance-level tooth segmentation extracts abundant localization and shape information from panoramic radiographs (PRs). The aim of this study was to evaluate the performance of a mask refinement network that extracts precise tooth edges. A public dataset which consists of 543 PRs and 16211 labelled teeth was utilized. The structure of a typical Mask Region-based Convolutional Neural Network (Mask RCNN) was used as the baseline. A novel loss function was designed focus on producing accurate mask edges. In addition to our proposed method, 3 existing tooth segmentation methods were also implemented on the dataset for comparative analysis. The average precisions (APs), mean intersection over union (mIoU), and mean Hausdorff distance (mHAU) were exploited to evaluate the performance of the network. A novel mask refinement region-based convolutional neural network was designed based on Mask RCNN architecture to extract refined masks for individual tooth on PRs. A total of 3311 teeth were correctly detected from 3382 tested teeth in 111 PRs. The AP, precision, and recall were 0.686, 0.979, and 0.952, respectively. Moreover, the mIoU and mHAU achieved 0.941 and 9.7, respectively, which are significantly better than the other existing segmentation methods. This study proposed an efficient deep learning algorithm for accurately extracting the mask of any individual tooth from PRs. Precise tooth masks can provide valuable reference for clinical diagnosis and treatment. This algorithm is a fundamental basis for further automated processing applications.

Identification of Unknown Marine Debris by ROVs Using Deep Learning and Different Convolutional Neural Network Structures

Identification of Unknown Marine Debris by ROVs Using Deep Learning and Different Convolutional Neural Network Structures

License plate Chinese character recognition based on ViT model

Transformer applications have been widely used in the computer vision field. Many related literatures show that the advantages of the model such as increased receptive field and globality are gradually emerging in image processing. However, with the popularity of the transformer, whether it can compete with the convolutional neural network (CNN) in terms of performance is still questionable and remains to be further studied. This paper will use the most basic structural model in the visual transformer (ViT) to classify and identify Chinese characters that are frequently used in the field of transportation and logistics and compare them with two classical CNN models. The results demonstrate that the performance of the transformer is obviously better than that of the traditional CNN structure, and the final accuracy of character recognition is higher than that of CNN, up to 98.66 %. It fully shows the infinite potential and excellent performance of the transformer in the area of computer vision and has high reliability and generalization ability.

CNN neural network temporal feature storage structure fusion for the visible channel equalization algorithm

The visible light communication channel has time-varying characteristics and is difficult to predict. This paper proposes an equalization algorithm based on the structure of a convolutional neural network (CNN), combining time series feature length and long short-term memory (LSTM), and adding a residual structure. It can be seen that the equalization coefficient vector of the optical channel is a time series, which can reflect the noise characteristics of the channel and has storage characteristics. The equalizer algorithm can accurately learn the complex channel characteristics and calculate the compensation coefficient according to the channel characteristics. It can also restore the original transmission signal. At the same time, this paper also examines the compensation method of the receiver in the mobile state. The long-term memory parameters of LSTM are used to represent the sequence causality in the memory channel, and CNN and residual structure are used to refine the results and improve the accuracy of the reconstruction. The simulation results show that the algorithm can effectively eliminate the influence of the fading characteristics of the visible optical channel, improve the bit error rate performance of system transmission, solve the overall problem of channel corruption, and precisely restore the original transmission signal with fast convergence speed. In addition, this method can achieve a better balance between performance and complexity compared to the traditional contention balancing method, which proves the potential and effectiveness of the proposed channel balancing method.

Ensemble of pre-learned deep learning model and an optimized LSTM for Alopecia Areata classification

Alopecia Areata (AA) is one of the most widespread diseases, which is generally classified and diagnosed by the Computer Aided Diagnosis (CAD) models. Though it improves AA diagnosis, it has limited interoperability and needs skilled radiologists in medical image interpretation. This problem can be solved by developing Deep Learning (DL) models with CAD for accurately diagnosing AA patients. Many studies engaged only in specific DL models such as Convolutional Neural Network (CNN) in medical imaging, which provides different independent results and many parameters, which limits their generalizability for different datasets. To combat this limitation, this work proposes an Ensemble Pre-Learned DL and an Optimized Long Short-Term Memory (EPL-OLSTM) model for AA classification. Initially, many healthy and AA scalp hair images are separately fed to the pre-learned CNN structures, i.e. AlexNet, ResNet, and InceptionNet to extract the deep features. Then, these features are passed to the OLSTM, in which the Battle Royale Optimization (BRO) algorithm is applied to optimize the LSTM’s hyperparameters. Moreover, the output of the LSTM is classified by the fuzzy-softmax into the associated AA classes, including mild, moderate, and severe. Thus, this model can increase the accuracy of differentiating between healthy and multiple AA scalp hair classes. Finally, an extensive experiment using the Figaro1k (for healthy scalp hair images) and DermNet (for different AA scalp hair images) datasets demonstrates that the EPL-OLSTM achieves 93.1% accuracy compared to the state-of-the-art DL models.

Dynamic Model-Embedded Intelligent Machine Fault Diagnosis Without Fault Data

Intelligent machine fault diagnosis technique has recently exploded interest in digital health, energy power, and industrial maintenance. Collecting machine fault data in engineering practice is usually costly, causing big challenges in the intelligent diagnosis of most fresh-from-the-factory machines that are missing fault data. Inspired by the main idea of the digital twin, this paper proposes a dynamic model-embedded intelligent machine fault diagnosis framework. Specifically, a machine dynamic modeling and parameter identification approach is introduced for building the machine digital model. The digital model is used to predict machine fault data from the measured healthy vibration signals. Furthermore, a parameterized convolutional neural network structure is designed for learning optimization features and recognizing the machine's healthy state. Experimental investigation demonstrates the effectiveness of the framework. The results show that the proposed framework enables intelligent machine fault diagnosis without fault data. The diagnosis effect outperforms supervised, unsupervised and small-sample learning approaches and can be generalized to another load and speed with acceptable accuracy.

Kernel Mapping Methods of Convolutional Neural Network in 3D NAND Flash Architecture

A flash memory is a non-volatile memory that has a large memory window, high cell density, and reliable switching characteristics and can be used as a synaptic device in a neuromorphic system based on 3D NAND flash architecture. We fabricated a TiN/Al2O3/Si3N4/SiO2/Si stack-based Flash memory device with a polysilicon channel. The input/output signals and output values are binarized for accurate vector-matrix multiplication operations in the hardware. In addition, we propose two kernel mapping methods for convolutional neural networks (CNN) in the neuromorphic system. The VMM operations of two mapping schemes are verified through SPICE simulation. Finally, the off-chip learning in the CNN structure is performed using the Modified National Institute of Standards and Technology (MNIST) dataset. We compared the two schemes in terms of various parameters and determined the advantages and disadvantages of each.