Traditional Deep Convolutional Neural Network Research Articles

An Algorithmic Study of Transformer-Based Road Scene Segmentation in Autonomous Driving

Applications such as autonomous driving require high-precision semantic image segmentation technology to identify and understand the content of each pixel in the images. Compared with traditional deep convolutional neural networks, the Transformer model is based on pure attention mechanisms, without convolutional layers or recurrent neural network layers. In this paper, we propose a new network structure called SwinLab, which is an improvement upon the Swin Transformer. Experimental results demonstrate that the improved SwinLab model achieves a segmentation accuracy comparable to that of deep convolutional neural network models in applications such as autonomous driving, with an MIoU of 77.61. Additionally, comparative experiments on the CityScapes dataset further validate the effectiveness and generalization of this structure. In conclusion, by refining the Swin Transformer, this paper simplifies the model structure, improves the training and inference speed, and maintains high accuracy, providing a more reliable semantic image segmentation solution for applications such as autonomous driving.

Deep learning model for diagnosis of thyroid nodules with size less than 1 cm: A multicenter, retrospective study

ObjectiveTo develop a ultrasound images based dual-channel deep learning model to achieve accurate early diagnosis of thyroid nodules less than 1 cm. MethodsA dual-channel deep learning model called thyroid nodule transformer network (TNT-Net) was proposed. The model has two input channels for transverse and longitudinal ultrasound images of thyroid nodules, respectively. A total of 9649 nodules from 8455 patients across five hospitals were retrospectively collected. The data were divided into a training set (8453 nodules, 7369 patients), an internal test set (565 nodules, 512 patients), and an external test set (631 nodules, 574 patients). ResultsTNT-Net achieved an area under the curve (AUC) of 0.953 (95 % confidence interval (CI): 0.934, 0.969) on the internal test set and 0.941 (95 % CI: 0.921, 0.957) on the external test set, significantly outperforming traditional deep convolutional neural network models and single-channel swin transformer model, whose AUCs ranged from 0.800 (95 % CI: 0.759, 0.837) to 0.856 (95 % CI: 0.819, 0.881). Furthermore, feature heatmap visualization showed that TNT-Net could extract richer and more energetic malignant nodule patterns. ConclusionThe proposed TNT-Net model significantly improved the recognition capability for thyroid nodules with size less than 1 cm. This model has the potential to reduce overdiagnosis and overtreatment of such nodules, providing essential support for precise management of thyroid nodules while complementing fine-needle aspiration biopsy.

DFC-Net: a dual-path frequency-domain cross-attention fusion network for retinal image quality assessment

Retinal image quality assessment (RIQA) is crucial for diagnosing various eye diseases and ensuring the accuracy of diagnostic analyses based on retinal fundus images. Traditional deep convolutional neural networks (CNNs) for RIQA face challenges such as over-reliance on RGB image brightness and difficulty in differentiating closely ranked image quality categories. To address these issues, we introduced the Dual-Path Frequency-domain Cross-attention Network (DFC-Net), which integrates RGB images and contrast-enhanced images using contrast-limited adaptive histogram equalization (CLAHE) as dual inputs. This approach improves structure detail detection and feature extraction. We also incorporated a frequency-domain attention mechanism (FDAM) to focus selectively on frequency components indicative of quality degradations and a cross-attention mechanism (CAM) to optimize the integration of dual inputs. Our experiments on the EyeQ and RIQA-RFMiD datasets demonstrated significant improvements, achieving a precision of 0.8895, recall of 0.8923, F1-score of 0.8909, and a Kappa score of 0.9191 on the EyeQ dataset. On the RIQA-RFMiD dataset, the precision was 0.702, recall 0.6729, F1-score 0.6869, and Kappa score 0.7210, outperforming current state-of-the-art approaches.

A two‐stage reactive power optimization method for distribution networks based on a hybrid model and data‐driven approach

AbstractThe uncertainty of distributed energy resources (DERs) and loads in distribution networks poses challenges for reactive power optimization and control timeliness. The computational limitations of the traditional algorithms and the development of artificial intelligence (AI) based technologies have promoted the advancement of hybrid model‐data‐driven algorithms. This article proposes a two‐stage reactive power optimization method for distributed networks (DNs) based on a hybrid model‐data‐driven approach. In the first stage, based on the topology and line parameters of the DN, as well as forecasts of loads and renewable energy outputs, a mixed‐integer second‐order cone programming (MISOCP) algorithm is used to control the on‐load tap changer (OLTC) positions on an hourly day‐ahead basis. In the second stage, leveraging deep learning technology, the real‐time reactive power output of photovoltaics (PV) and wind power units is controlled at a 5‐min time scale throughout the day. Specifically, using traditional solvers, the global optimal reactive power output for PV and wind power units is determined first, corresponding to various load and renewable energy output scenarios. Then, neural networks are trained to map node power to the optimal reactive power outputs of renewable energy units, capturing the complex physical relationships. For the second stage, a transformer network framework with a self‐attention mechanism and multi‐head attention for deep learning training is applied to uncover the intrinsic and physical spatial relationships among high‐dimensional features. The proposed method is tested on a modified IEEE 33‐bus system with multiple distributed renewable energy sources. The case study results demonstrate that the proposed hybrid model‐data‐driven algorithm effectively coordinates day‐ahead and real‐time controls of various devices, achieving real‐time model‐free optimization throughout the day. Compared to traditional deep neural networks (DNNs) and convolutional neural networks (CNNs), the transformer network provides superior reactive power optimization results.

CoreViT: A new vision transformer model for lithofacies identification in cores

Lithofacies identification is crucial in fields such as geology, geotechnical engineering, rock mechanics, and petroleum engineering, as it reveals the physical, chemical, and mineralogical characteristics of rocks that aid in identifying oil and gas reservoir types, volumes, and productivity. Traditional methods of identifying rock types, such as manual sample identification, micrograph identification, and experimental identification, are costly and time-consuming. Several studies have endeavored to enhance the degree of lithofacies identification automation using machine learning (ML) and deep learning (DL) techniques. However, their capacity to recognize intricate data remains restricted. To overcome this challenge, we introduce CoreViT, a modified architecture of the advanced vision transformer (ViT) that includes the parallel transformer encoder (PTE) for exchanging information among image patches, and the class encoder (CE) for improved automatic lithofacies identification. Our study, focusing on core images including algal limestone, mudstone, and non-algal limestone from the Fengxi Well 1 in the Qaidam Basin, demonstrates CoreViT's average recognition accuracy at approximately 97.5% with an average loss of 0.071. Compared with other classical convolutional neural network (CNN) models, CoreViT achieves relatively high accuracy and low loss. This study highlights the superiority of the ViT model over traditional deep convolutional neural networks (DCNNs) and suggests the great potentiality of applying the ViT model in lithofacies identification in core samples.

An effective colorectal polyp classification for histopathological images based on supervised contrastive learning

Early detection of colon adenomatous polyps is pivotal in reducing colon cancer risk. In this context, accurately distinguishing between adenomatous polyp subtypes, especially tubular and tubulovillous, from hyperplastic variants is crucial. This study introduces a cutting-edge computer-aided diagnosis system optimized for this task. Our system employs advanced Supervised Contrastive learning to ensure precise classification of colon histopathology images. Significantly, we have integrated the Big Transfer model, which has gained prominence for its exemplary adaptability to visual tasks in medical imaging. Our novel approach discerns between in-class and out-of-class images, thereby elevating its discriminatory power for polyp subtypes. We validated our system using two datasets: a specially curated one and the publicly accessible UniToPatho dataset. The results reveal that our model markedly surpasses traditional deep convolutional neural networks, registering classification accuracies of 87.1% and 70.3% for the custom and UniToPatho datasets, respectively. Such results emphasize the transformative potential of our model in polyp classification endeavors.

A novel approach for intelligent diagnosis and grading of diabetic retinopathy

Diabetic retinopathy (DR) is a severe ocular complication of diabetes that can lead to vision damage and even blindness. Currently, traditional deep convolutional neural networks (CNNs) used for DR grading tasks face two primary challenges: (1) insensitivity to minority classes due to imbalanced data distribution, and (2) neglecting the relationship between the left and right eyes by utilizing the fundus image of only one eye for training without differentiating between them. To tackle these challenges, we proposed the DRGCNN (DR Grading CNN) model. To solve the problem caused by imbalanced data distribution, our model adopts a more balanced strategy by allocating an equal number of channels to feature maps representing various DR categories. Furthermore, we introduce a CAM-EfficientNetV2-M encoder dedicated to encoding input retinal fundus images for feature vector generation. The number of parameters of our encoder is 52.88 M, which is less than RegNet_y_16gf (80.57 M) and EfficientNetB7 (63.79 M), but the corresponding kappa value is higher. Additionally, in order to take advantage of the binocular relationship, we input fundus retinal images from both eyes of the patient into the network for features fusion during the training phase. We achieved a kappa value of 86.62% on the EyePACS dataset and 86.16% on the Messidor-2 dataset. Experimental results on these representative datasets for diabetic retinopathy (DR) demonstrate the exceptional performance of our DRGCNN model, establishing it as a highly competitive intelligent classification model in the field of DR. The code is available for use at https://github.com/Fat-Hai/DRGCNN.

Art appreciation model design based on improved PageRank and ECA-ResNeXt50 algorithm.

Image sentiment analysis technology can predict, measure and understand the emotional experience of human beings through images. Aiming at the problem of extracting emotional characteristics in art appreciation, this article puts forward an innovative method. Firstly, the PageRank algorithm is enhanced using tweet content similarity and time factors; secondly, the SE-ResNet network design is used to integrate Efficient Channel Attention (ECA) with the residual network structure, and ResNeXt50 is optimized to enhance the extraction of image sentiment features. Finally, the weight coefficients of overall emotions are dynamically adjusted to select a specific emotion incorporation strategy, resulting in effective bimodal fusion. The proposed model demonstrates exceptional performance in predicting sentiment labels, with maximum classification accuracy reaching 88.20%. The accuracy improvement of 21.34% compared to the traditional deep convolutional neural networks (DCNN) model attests to the effectiveness of this study. This research enriches images and texts' emotion feature extraction capabilities and improves the accuracy of emotion fusion classification.

A segmentation model to detect cevical lesions based on machine learning of colposcopic images

BackgroundSemantic segmentation is crucial in medical image diagnosis. Traditional deep convolutional neural networks excel in image classification and object detection but fall short in segmentation tasks. Enhancing the accuracy and efficiency of detecting high-level cervical lesions and invasive cancer poses a primary challenge in segmentation model development. MethodsBetween 2018 and 2022, we retrospectively studied a total of 777 patients, comprising 339 patients with high-level cervical lesions and 313 patients with microinvasive or invasive cervical cancer. Overall, 1554 colposcopic images were put into the DeepLabv3+ model for learning. Accuracy, Precision, Specificity, and mIoU were employed to evaluate the performance of the model in the prediction of cervical high-level lesions and cancer. ResultsExperiments showed that our segmentation model had better diagnosis efficiency than colposcopic experts and other artificial intelligence models, and reached Accuracy of 93.29 %, Precision of 87.2 %, Specificity of 90.1 %, and mIoU of 80.27 %, respectively. ConclutionThe DeepLabv3+ model had good performance in the segmentation of cervical lesions in colposcopic post-acetic-acid images and can better assist colposcopists in improving the diagnosis.

New Hybrid Graph Convolution Neural Network with Applications in Game Strategy

Deep convolutional neural networks (DCNNs) have enjoyed much success in many applications, such as computer vision, automated medical diagnosis, autonomous systems, etc. Another application of DCNNs is for game strategies, where the deep neural network architecture can be used to directly represent and learn strategies from expert players on different sides. Many game states can be expressed not only as a matrix data structure suitable for DCNN training but also as a graph data structure. Most of the available DCNN methods ignore the territory characteristics of both sides’ positions based on the game rules. Therefore, in this paper, we propose a hybrid approach to the graph neural network to extract the features of the model of game-playing strategies and fuse it into a DCNN. As a graph learning model, graph convolutional networks (GCNs) provide a scheme by which to extract the features in a graph structure, which can better extract the features in the relationship between the game-playing strategies. We validate the work and design a hybrid network to integrate GCNs and DCNNs in the game of Go and show that on the KGS Go dataset, the performance of the hybrid model outperforms the traditional DCNN model. The hybrid model demonstrates a good performance in extracting the game strategy of Go.

An Efficient, Lightweight, Tiny 2D-CNN Ensemble Model to Detect Cardiomegaly in Heart CT Images.

Cardiomegaly is a significant global health concern, especially in developing nations. Although advanced clinical care is available for newly diagnosed patients, many in resource-limited regions face late diagnoses and consequent increased mortality. This challenge is accentuated by a scarcity of radiography equipment and radiologists. Hence, we propose the development of a computer-aided diagnostic (CAD) system, specifically a lightweight, tiny 2D-CNN ensemble model, to facilitate early detection and, potentially, reduce mortality rates. Deep learning, with its subset of convolutional neural networks (CNN), has shown potential in visual applications, especially in medical image diagnosis. However, traditional deep CNNs often face compatibility issues with object-oriented human factor technology. Our proposed model aims to bridge this gap. Using CT scan images sourced from the Mendeley data center, our tiny 2D-CNN ensemble learning model achieved an accuracy of 96.32%, offering a promising tool for efficient and accurate cardiomegaly detection.

On the Imaginary Wings: Text-Assisted Complex-Valued Fusion Network for Fine-Grained Visual Classification.

Fine-grained visual classification (FGVC) is challenging due to the interclass similarity and intraclass variation in datasets. In this work, we explore the great merit of complex values in introducing an imaginary part for modeling data uncertainty (e.g., different points on the complex plane can describe the same state) and graph convolutional networks (GCNs) in learning interdependently among classes to simultaneously tackle the above two major challenges. To the end, we propose a novel approach, termed text-assisted complex-valued fusion network (TA-CFN). Specifically, we expand each feature from 1-D real values to 2-D complex value by disassembling feature maps, thereby enabling the extension of traditional deep convolutional neural networks over the complex domain. Then, we fuse the real and imaginary parts of complex features through complex projection and modulus operation. Finally, we build an undirected graph over the object labels with the assistance of a text corpus, and a GCN is learned to map this graph into a set of classifiers. The benefits are in two folds: 1) complex features allow for a richer algebraic structure to better model the large variation within the same category and 2) leveraging the interclass dependencies brought by the GCN to capture key factors of the slight variation among different categories. We conduct extensive experiments to verify that our proposed model can achieve the state-of-the-art performance on two widely used FGVC datasets.

Optimal Design of Antennas based on CNN and BLS

Surrogate can greatly reduce the computing cost of full-wave electromagnetic simulation software in antenna design, and neural networks as a kind of surrogate have become very popular in recent years, among which the Convolutional Neural Network (CNN) is a very hot topic due to its better feature extraction and generalization capability. In this paper, Broad Learning System (BLS) is introduced to address the problems that CNN may fall into with local optimization and overfitting when dealing with antenna data. It provides a learning method to increase nodes in width for regression prediction, which has good generalization ability. Specifically, in antenna design, feature extraction is first applied using the convolution and pooling of CNN. The original dataset is combined with the extracted feature data to realize data augmentation, and then the BLS is used for regression training. Compared with the traditional deep CNN, this model uses lightweight convolutional layers, which is promising in solving the problem of the network falling into local optimum or overfitting. The simulation results show that the accuracy reaches 93.91% and 92.34% respectively on the Abalone benchmark and antenna data, proving the effectiveness of the proposed model in this paper.

Triplet-Net Classification of Contiguous Stem Cell Microscopy Images.

Cellular microscopy imaging is a common form of data acquisition for biological experimentation. Observation of gray-level morphological features allows for the inference of useful biological information such as cellular health and growth status. Cellular colonies can contain multiple cell types, making colony level classification very difficult. Additionally, cell types growing in a hierarchical, downstream fashion, can often look visually similar, although biologically distinct. In this paper, it is determined empirically that traditional deep Convolutional Neural Networks (CNN) and classical object recognition techniques are not sufficient to distinguish between these subtle visual differences, resulting in misclassifications. Instead, Triplet-net CNN learning is employed in a hierarchical classification scheme to improve the ability of the model to discern distinct, fine-grain features of two commonly confused morphological image-patch classes, namely Dense and Spread colonies. The Triplet-net method improves classification accuracy over a four-class deep neural network by ∼ 3 %, a value that was determined to be statistically significant, as well as existing state-of-the-art image patch classification approaches and standard template matching. These findings allow for the accurate classification of multi-class cell colonies with contiguous boundaries, and increased reliability and efficiency of automated, high-throughput experimental quantification using non-invasive microscopy.

BiTNet: Hybrid deep convolutional model for ultrasound image analysis of human biliary tract and its applications

Certain life-threatening abnormalities, such as cholangiocarcinoma, in the human biliary tract are curable if detected at an early stage, and ultrasonography has been proven to be an effective tool for identifying them. However, the diagnosis often requires a second opinion from experienced radiologists, who are usually overwhelmed by many cases. Therefore, we propose a deep convolutional neural network model, named biliary tract network (BiTNet), developed to solve problems in the current screening system and to avoid overconfidence issues of traditional deep convolutional neural networks. Additionally, we present an ultrasound image dataset for the human biliary tract and demonstrate two artificial intelligence (AI) applications: auto-prescreening and assisting tools. The proposed model is the first AI model to automatically screen and diagnose upper-abdominal abnormalities from ultrasound images in real-world healthcare scenarios. Our experiments suggest that prediction probability has an impact on both applications, and our modifications to EfficientNet solve the overconfidence problem, thereby improving the performance of both applications and of healthcare professionals. The proposed BiTNet can reduce the workload of radiologists by 35% while keeping the false negatives to as low as 1 out of every 455 images. Our experiments involving 11 healthcare professionals with four different levels of experience reveal that BiTNet improves the diagnostic performance of participants of all levels. The mean accuracy and precision of the participants with BiTNet as an assisting tool (0.74 and 0.61, respectively) are statistically higher than those of participants without the assisting tool (0.50 and 0.46, respectively (p<0.001)). These experimental results demonstrate the high potential of BiTNet for use in clinical settings.

A Rapid Identification Technique of Moving Loads Based on MobileNetV2 and Transfer Learning

Rapid and accurate identification of moving load is crucial for bridge operation management and early warning of overload events. However, it is hard to obtain them rapidly via traditional machine learning methods, due to their massive model parameters and complex network structure. To this end, this paper proposes a novel method to perform moving loads identification using MobileNetV2 and transfer learning. Specifically, the dynamic responses of a vehicle–bridge interaction system are firstly transformed into a two-dimensional time-frequency image by continuous wavelet transform to construct the database. Secondly, a pre-trained MobileNetV2 model based on ImageNet is transferred to the moving load identification task by transfer learning strategy for describing the mapping relationship between structural response and these specified moving loads. Then, load identification can be performed through inputting bridge responses into the established relationship. Finally, the effectiveness of the method is verified by numerical simulation. The results show that it can accurately identify the vehicle weight, vehicle speed information, and presents excellent strong robustness. In addition, MobileNetV2 has faster identification speed and requires less computational resources than several traditional deep convolutional neural network models in moving load identification, which can provide a novel idea for the rapid identification of moving loads.

HC-Net: A hybrid convolutional network for non-human primate brain extraction.

Brain extraction (skull stripping) is an essential step in the magnetic resonance imaging (MRI) analysis of brain sciences. However, most of the current brain extraction methods that achieve satisfactory results for human brains are often challenged by non-human primate brains. Due to the small sample characteristics and the nature of thick-slice scanning of macaque MRI data, traditional deep convolutional neural networks (DCNNs) are unable to obtain excellent results. To overcome this challenge, this study proposed a symmetrical end-to-end trainable hybrid convolutional neural network (HC-Net). It makes full use of the spatial information between adjacent slices of the MRI image sequence and combines three consecutive slices from three axes for 3D convolutions, which reduces the calculation consumption and promotes accuracy. The HC-Net consists of encoding and decoding structures of 3D convolutions and 2D convolutions in series. The effective use of 2D convolutions and 3D convolutions relieves the underfitting of 2D convolutions to spatial features and the overfitting of 3D convolutions to small samples. After evaluating macaque brain data from different sites, the results showed that HC-Net performed better in inference time (approximately 13 s per volume) and accuracy (mean Dice coefficient reached 95.46%). The HC-Net model also had good generalization ability and stability in different modes of brain extraction tasks.

A Modeling Design Method for Complex Products Based on LSTM Neural Network and Kansei Engineering

Complex products (CPs) modeling design has a long development cycle and high cost, and it is difficult to accurately meet the needs of enterprises and users. At present, the Kansei Engineering (KE) method based on back-propagated (BP) neural networks is applied to solve the modeling design problem that meets users’ affective preferences for simple products quickly and effectively. However, the modeling feature data of CPs have a wide range of dimensions, long parameter codes, and the characteristics of time series. As a result, it is difficult for BP neural networks to recognize the affective preferences of CPs from an overall visual perception level as humans do. To address the problems above and assist designers with efficient and high-quality design, a CP modeling design method based on Long Short-Term Memory (LSTM) neural network and KE (CP-KEDL) was proposed. Firstly, the improved MA method was carried out to transform the product modeling features into feature codes with sequence characteristics. Secondly, the mapping model between perceptual images and modeling features was established based on the LSTM neural network to predict the evaluation value of the product’s perceptual images. Finally, the optimal feature sets were calculated by a Genetic Algorithm (GA). The experimental results show that the MSE of the LSTM model is only 0.02, whereas the MSE of the traditional Deep Neural Networks (DNN) and Convolutional Neural Networks (CNN) neural network models are 0.30 and 0.23, respectively. The results verified that the proposed method can effectively grapple with the CP modeling design problem with the timing factor, improve design satisfaction and shorten the R&amp;D cycle of CP industrial design.

Adaptive Threshold Hierarchical Incremental Learning Method

Traditional deep convolutional neural networks have achieved excellent performance on various machine learning tasks. Still, they perform poorly in continuous data stream environments, where models trained on new datasets often suffer from a significant drop in performance on old datasets, a phenomenon known as “catastrophic forgetting.” Incremental learning can help solve the “catastrophic forgetting” problem in deep learning by learning new knowledge while retaining what has already been learned. In practice, incremental learning algorithms usually need to be deployed on edge devices with limited memory and restricted access to training data, facing the problems of high model complexity and imbalance between old and new categories of data. We propose an Adaptive Threshold Hierarchical Incremental Learning (ATHIL) method to address the above problems. Our proposed method does not require additional data and model storage space during the training process, combines local weight discrete coefficient thresholding and the mean nearest neighbor principle, uses a sparse matrix hierarchical masking network, and flexibly adjusts the network structure according to different tasks to achieve learning multiple image classification tasks in a single network. The experimental results show that the performance of the proposed method significantly outperforms existing methods on fine-grained classification datasets under three evaluation metrics.

Improved Wafer Map Inspection Using Attention Mechanism and Cosine Normalization

Wafer map inspection is essential for semiconductor manufacturing quality control and analysis. The deep convolutional neural network (DCNN) is the most effective algorithm in wafer defect pattern analysis. Traditional DCNNs rely heavily on high quality datasets for training. However, obtaining balanced and sufficient labeled data is difficult in practice. This paper reconsiders the causes of the imbalance and proposes a deep learning method that can learn robust knowledge from an imbalanced dataset using the attention mechanism and cosine normalization. We interpret the dataset imbalance as both a feature and a quantity distribution imbalance. To compensate for feature distribution imbalance, we add an improved convolutional attention module to the DCNN to enhance representation. In particular, a feature-map-specific direction mapping module is developed to amplify the positional information of defect clusters. For quantity distribution imbalance, the cosine normalization algorithm is proposed to replace the fully connected layer, and classifier fine-tuning is realized through a small amount of iterative training, which decreases the sensitivity to the quantitative distribution. The experimental results on real-world datasets demonstrate that the proposed method significantly improves the robustness of wafer map inspection and outperforms existing algorithms when trained on imbalanced datasets.