Convolutional Neural Network-based Method Research Articles

Visual Place Recognition via a Multitask Learning Method With Attentive Feature Aggregation

Visual place recognition has gained popularity in recent years. Mainstream convolutional neural network-based methods formulate it as a ranking task and optimize it in the paradigm of deep metric learning, however, the ranking-motivated losses concern only the ranking relationship for each query image and the compactness of intra-place feature distribution is ignored. It is still challenging due to varying viewpoints, illuminations and even dynamic objects. In this paper, a novel multi-task learning framework is proposed, which combines the existing triplet ranking task and our designed binary classification task to jointly optimize the network for better generalization capability. Specifically, a binary classification network with the corresponding binary cross-entropy loss is designed in the classification task. In this way, the intra-place feature compactness and inter-place feature separability are reinforced. At testing stage, this classification network is discarded without increasing the computation cost. Furthermore, an attention module is presented to promote the network to concentrate on the salient regions by assigning different importance to each spatial position. Our method achieves the top-10 recalls of 97.27%, 94.6%, and 96.93% on Pitts250k-test, Tokyo 24/7, and TokyoTM-val datasets, respectively. Extensive experiments prove that the proposed network can learn discriminative global features with better robustness to viewpoints and environmental variations.

Photo-zSNthesis: Converting Type Ia Supernova Lightcurves to Redshift Estimates via Deep Learning

Upcoming photometric surveys will discover tens of thousands of Type Ia supernovae (SNe Ia), vastly outpacing the capacity of our spectroscopic resources. In order to maximize the scientific return of these observations in the absence of spectroscopic information, we must accurately extract key parameters, such as SN redshifts, with photometric information alone. We present Photo-zSNthesis, a convolutional neural network-based method for predicting full redshift probability distributions from multi-band supernova lightcurves, tested on both simulated Sloan Digital Sky Survey (SDSS) and Vera C. Rubin Legacy Survey of Space and Time data as well as observed SDSS SNe. We show major improvements over predictions from existing methods on both simulations and real observations as well as minimal redshift-dependent bias, which is a challenge due to selection effects, e.g., Malmquist bias. Specifically, we show a 61× improvement in prediction bias 〈Δz〉 on PLAsTiCC simulations and 5× improvement on real SDSS data compared to results from a widely used photometric redshift estimator, LCFIT+Z. The PDFs produced by this method are well constrained and will maximize the cosmological constraining power of photometric SNe Ia samples.

Graph neural network method for the intelligent selection of river system

The spatial features and generalisation rules for river network generalisation are difficult to directly quantify using indicators. To consider dimensional information hidden in river networks and improve river network selection accuracy, this study introduces a graph convolutional neural network-based method. First, we modelled the river network as a graph structure, where the nodes represent each river segment and edges represent the connections between river segments. The semantic, geometric, and morphological features of individual river segments and topological and constraint features between river segments were then calculated to characterise the relevant nodes. Second, under supervised classification, the input node attributes and labels were sampled and aggregated to obtain richer and more abstract high-level features. The graph convolutional neural network model then selected or deleted river segments. Finally, the selected individual river segments were connected to obtain the complete integrated river network. A 1:10,000 scale map of the Min River system in the Yangtze River Basin was tested, with a 1:50,000 scale map used as the control, and it yielded a correct classification rate >95%. Moreover, the correct classification rate was 7.35%–5.31% and 7.7%–3.3% higher than that of other graph neural network methods and traditional machine learning methods, respectively.

Pyramid Attention Network for Image Restoration

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their “clean” correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks

Convolutional neural network-based recognition method for volleyball movements

With the development of network technology and computer intelligent monitoring technology, a large number of video data came into being. In view of the analysis of specific targets in video, the traditional artificial analysis method can not meet the existing needs. In volleyball teaching in college physical education, because each student has different movements, intelligent processing of video data has become a key issue. Through collecting and sorting out the relevant research results on behavior recognition, it is found that in the research of deep learning, the algorithm structure applied in this paper is representative, and its strong learning ability, especially compared with the traditional algorithm, can more accurately identify human movements, natural language processing and so on, but the research on volleyball action recognition is still less. Therefore, this paper constructs a data set, improves the convolution neural network model, subsequently, new models are constructed through the neural network structure to improve the accuracy of the nonlinear expression and optimize the content of the input data. In order to more accurately analyze the effectiveness of this algorithm, the new data are obtained by grouping the volleyball games in college sports courses. Compared with the original paper, the accuracy of the improved 3D network is improved by 3.3%–88.5%, and the complexity is reduced by 33.6%.

Estimating the State of Health for Lithium-ion Batteries: A Particle Swarm Optimization-Assisted Deep Domain Adaptation Approach

The state of health (SOH) is a critical factor in evaluating the performance of the lithium-ion batteries (LIBs). Due to various end-user behaviors, the LIBs exhibit different degradation modes, which makes it challenging to estimate the SOHs in a personalized way. In this article, we present a novel particle swarm optimization-assisted deep domain adaptation (PSO-DDA) method to estimate the SOH of LIBs in a personalized manner, where a new domain adaptation strategy is put forward to reduce cross-domain distribution discrepancy. The standard PSO algorithm is exploited to automatically adjust the chosen hyperparameters of developed DDA-based method. The proposed PSODDA method is validated by extensive experiments on two LIB datasets with different battery chemistry materials, ambient temperatures and charge-discharge configurations. Experimental results indicate that the proposed PSO-DDA method surpasses the convolutional neural network-based method and the standard DDA-based method. The PyTorch implementation of the proposed PSO-DDA method is available at https://github.com/mxt0607/PSO-DDA.

A Hybrid Approach for retinal image super-resolution

Experts require large high-resolution retinal images to detect tiny abnormalities, such as microaneurysms or issues of vascular branches. However, these images often suffer from low quality (e.g., resolution) due to poor imaging device configuration and misoperations. Many works utilized Convolutional Neural Network-based (CNN) methods for image super-resolution. The authors focused on making these models more complex by adding layers and various blocks. It leads to additional computational expenses and obstructs the application in real-life scenarios. Thus, this paper proposes a novel, lightweight, deep-learning super-resolution method for retinal images. It comprises a Vision Transformer (ViT) encoder and a convolutional neural network decoder. To our best knowledge, this is the first attempt to use a transformer-based network to solve the issue of accurate retinal image super-resolution. A progressively growing super-resolution training technique is applied to increase the resolution of images by factors of 2, 4, and 8. The prominent architecture remains constant thanks to the adaptive patch embedding layer, which does not lead to additional computational expense due to increased up-scaling factors. This patch embedding layer includes 2-dimensional convolution with specific values of kernel size and strides that depend on the input shape. This strategy has removed the need to append additional super-resolution blocks to the model. The proposed method has been evaluated through quantitative and qualitative measures. The qualitative analysis also includes vessel segmentation of super-resolved and ground truth images. Experimental results indicate that the proposed method outperforms the current state-of-the-art methods.

High-Resolution Network with Dynamic Convolution and Coordinate Attention for Classification of Chest X-ray Images.

The development of automatic chest X-ray (CXR) disease classification algorithms is significant for diagnosing thoracic diseases. Owing to the characteristics of lesions in CXR images, including high similarity in appearance of the disease, varied sizes, and different occurrence locations, most existing convolutional neural network-based methods have insufficient feature extraction for thoracic lesions and struggle to adapt to changes in lesion size and location. To address these issues, this study proposes a high-resolution classification network with dynamic convolution and coordinate attention (HRCC-Net). In the method, this study suggests a parallel multi-resolution network in which a high-resolution branch acquires essential detailed features of the lesion and multi-resolution feature swapping and fusion to obtain multiple receptive fields to extract complicated disease features adequately. Furthermore, this study proposes dynamic convolution to enhance the network's ability to represent multi-scale information to accommodate lesions of diverse scales. In addition, this study introduces a coordinate attention mechanism, which enables automatic focus on pathologically relevant regions and capturing the variations in lesion location. The proposed method is evaluated on ChestX-ray14 and CheXpert datasets. The average AUC (area under ROC curve) values reach 0.845 and 0.913, respectively, indicating this method's advantages compared with the currently available methods. Meanwhile, with its specificity and sensitivity to measure the performance of medical diagnostic systems, the network can improve diagnostic efficiency while reducing the rate of misdiagnosis. The proposed algorithm has great potential for thoracic disease diagnosis and treatment.

Fast Non-Local Attention network for light super-resolution

Although convolutional neural network-based methods have achieved significant performance improvement for Single Image Super-Resolution (SISR), their vast computational cost hinders real-world environment application. Thus, the interest in light networks for SISR is rising. Since existing SISR light models mainly focus on extracting fine local features using convolution operation, they have a limitation in that networks hardly capture global information. To capture the long-range dependency, Non-Local (NL) attention and Transformers have been explored in the SISR task. However, they are still suffering from a balancing problem between performance and computational cost. In this paper, we propose Fast Non-Local attention NETwork (FNLNET) for a super light SISR, which can capture the global representation. To acquire global information, we propose The Fast Non-Local Attention (FNLA) module that has low computational complexity while capturing global representation that reflects long-distance relationships between patches. Then, FNLA requires only 16 times lower computational cost than conventional NL networks while improving performance. In addition, we propose a powerful module called Global Self-Intension Mining (GSIM) that fuses the multi-information resources such as local, and global representation. Our FNLNET shows outstanding performance with fewer parameters and computational costs in the experiments on the benchmark datasets against state-of-the-art light SISR models.

A Large-scale Virtual Dataset and Egocentric Localization for Disaster Responses.

With the increasing social demands of disaster response, methods of visual observation for rescue and safety have become increasingly important. However, because of the shortage of datasets for disaster scenarios, there has been little progress in computer vision and robotics in this field. With this in mind, we present the first large-scale synthetic dataset of egocentric viewpoints for disaster scenarios. We simulate pre- and post-disaster cases with drastic changes in appearance, such as buildings on fire and earthquakes. The dataset consists of more than 300K high-resolution stereo image pairs, all annotated with ground-truth data for the semantic label, depth in metric scale, optical flow with sub-pixel precision, and surface normal as well as their corresponding camera poses. To create realistic disaster scenes, we manually augment the effects with 3D models using physically-based graphics tools. We train various state-of-the-art methods to perform computer vision tasks using our dataset, evaluate how well these methods recognize the disaster situations, and produce reliable results of virtual scenes as well as real-world images. We also present a convolutional neural network-based egocentric localization method that is robust to drastic appearance changes, such as the texture changes in a fire, and layout changes from a collapse. To address these key challenges, we propose a new model that learns a shape-based representation by training on stylized images, and incorporate the dominant planes of query images as approximate scene coordinates. We evaluate the proposed method using various scenes including a simulated disaster dataset to demonstrate the effectiveness of our method when confronted with significant changes in scene layout. Experimental results show that our method provides reliable camera pose predictions despite vastly changed conditions.

A CNN-Based Layer-Adaptive GCPs Extraction Method for TIR Remote Sensing Images

Ground Control Points (GCPs) are of great significance for applications involving the registration and fusion of heterologous remote sensing images (RSIs). However, utilizing low-level information rather than deep features, traditional methods based on intensity and local image features turn out to be unsuitable for heterologous RSIs because of the large nonlinear radiation difference (NRD), inconsistent resolutions, and geometric distortions. Additionally, the limitations of current heterologous datasets and existing deep-learning-based methods make it difficult to obtain enough precision GCPs from different kinds of heterologous RSIs, especially for thermal infrared (TIR) images that present low spatial resolution and poor contrast. In this paper, to address the problems above, we propose a convolutional neural network-based (CNN-based) layer-adaptive GCPs extraction method for TIR RSIs. Particularly, the constructed feature extraction network is comprised of basic and layer-adaptive modules. The former is used to achieve the coarse extraction, and the latter is designed to obtain high-accuracy GCPs by adaptively updating the layers in the module to capture the fine communal homogenous features of the heterologous RSIs until the GCP precision meets the preset threshold. Experimental results evaluated on TIR images of SDGSAT-1 TIS and near infrared (NIR), short wave infrared (SWIR), and panchromatic (PAN) images of Landsat-8 OLI show that the matching root-mean-square error (RMSE) of TIS images with SWIR and NIR images could reach 0.8 pixels and an even much higher accuracy of 0.1 pixels could be reached between TIS and PAN images, which performs better than those of the traditional methods, such as SIFT, RIFT, and the CNN-based method like D2-Net.

A novel one-dimensional convolutional neural network-based method for emotion recognition of electric power industry workers

When the workers of electric power industry are going to perform high-risk or dispatching operations, it is important to assess the emotional state of the workers. Emotion recognition is helpful to justify whether the works are suitable to grasp and handle the operation status of the power system in time. In this paper, we develop a simple and efficient automatic emotion recognition method based on deep learning theory using ECG signals. Firstly, since Z-score method which is quite popular for emotion recognition exists high missed outlier detection rate, we define a basic beat and removed the abnormal beats according to the similarities between the truncated beats and the basic beat. Secondly, in order to make it easily for Convolutional Neural Network (CNN) to extract the periodicity features of one-dimensional signal in time domain, we present stacking operation of the truncated segments to increase the dimension of ECG data from one to three. Finally, aiming at decreasing the computation burden, a one-dimensional CNN only consisted of two convolution blocks is designed. The results of applying the proposed approach to the benchmark Wearable Stress and Affect Data set (WESAD) demonstrate that our proposal is capable of classifying four different kinds of emotions with an averaged accuracy of 98.57% and F1 value of 98.73%, respectively. Compared against some literature, the performance of our proposal is superior to them.

USING CONVOLUTIONAL NEURAL NETWORK-BASED SEGMENTATION FOR IMAGE-BASED COMPUTATIONAL FLUID DYNAMICS SIMULATIONS OF BRAIN ANEURYSMS: INITIAL EXPERIENCE IN AUTOMATED MODEL CREATION.

"Image-based" computational fluid dynamics (CFD) simulations provide insights into each patient's hemodynamic environment. However, current standard procedures for creating CFD models start with manual segmentation and are time-consuming, hindering the clinical translation of image-based CFD simulations. This feasibility study adopts deep-learning-based image segmentation (hereafter referred to as Artificial Intelligence (AI) segmentation) to replace manual segmentation to accelerate CFD model creation. Two published convolutional neural network-based AI methods (MIScnn and DeepMedic) were selected to perform CFD model extraction from three-dimensional (3D) rotational angiography data containing intracranial aneurysms. In this study, aneurysm morphological and hemodynamic results using models generated by AI segmentation methods were compared with those obtained by two human users for the same data. Interclass coefficients (ICC), Bland-Altman plots, and Pearson's correlation coefficients (PCC) were combined to assess how well AI-generated CFD models were performed. We found that almost perfect agreement was obtained between the human and AI results for all eleven morphological and five out of eight hemodynamic parameters, while a moderate agreement was obtained from the remaining three hemodynamic parameters. Given this level of agreement, using AI segmentation to create CFD models is feasible, given more developments.

Study of convolutional neural network-based semantic segmentation methods on edge intelligence devices for field agricultural robot navigation line extraction

Smart agricultural machinery is emerging as the future trend for field robots, and the fully automatic robot has a great application prospect. However, it is a big challenge for robots to navigate in complex farmland environments. In this research, 5 deep learning-based computer vision methods under different field scenes for field navigation line extraction were studied and successfully deployed on an embedded system, which can be integrated into robots for automatic navigation in the future. The field road was segmented by the semantic segmentation algorithm at first, and then the navigation line is extracted from the segmented image by a polygon fitting method. Finally, all the models are transformed through the TensorRT library and deployed on the edge computing device Jetson Nano. In the experiment, five reprehensive semantic segmentation networks namely UNet, Deeplabv3+, BiseNetv1, BiseNetv2, and ENet networks were selected. Among the five networks, Deeplabv3+ is the most accurate. In five scenes, its average segmentation accuracy is 84.87 %, and the navigation line error is 9.59 pixels. Especially in the third scene with shadow and occlusion, it performs best, with only 8.34 pixel error, But the speed of Deeplabv3+ is only 9.7 FPS. ENet, BiseNetv1, and BiseNetv2 are lightweight networks. The speed of ENet is 16.8 FPS, BiseNetv2 is 17 FPS, and BiseNetv1 is 15.8 FPS. In segmentation accuracy and navigation line error, ENet performs better than BiseNet series networks, which are 84.94 % and 10.73 pixels, respectively. In the third scene with shadow and occlusion, it also performs slightly better than BiseNet series networks. In summary, deep learning-based semantic segmentation methods have strong robustness and stability in complex environment compared with previous research. Among all currently available neural networks, ENet has the best performance and good application potential in field navigation.

A SURVEY ON AIR WRITING CHARACTER RECOGNITION AND TRANSLATION

Computer systems are increasingly using airwriting recognition since it can be done quickly and intelligently. The term "air-writing acknowledgment apparatus" refers to the use of a stylus to draw an articulated character or expression in free space. This study presents a simple yet effective convolutional neural network-based air-writing recognition method (CNN). It is suggested to use a plausible algorithm tracking hands to reward air-writing circles captured through a web camera. This study provides a straightforward but efficient method for recognizing air-writing that uses deep convolutional neural networks (CNNs). With the aid of hand motions, users can write characters in the air that are subsequently translated into written text using an air writing character recognition and translation technology. Convolutional neural networks (CNN), which are trained to recognise and decipher human hand gestures, power this technology. The device uses a camera to record the hand movements and then runs the data through CNN to identify the characters being typed. The identified characters are then converted into text that may be presented on a screen using text-to-speech technology.

A One-Dimensional Convolutional Neural Network-Based Method for Diagnosis of Tooth Root Cracks in Asymmetric Spur Gear Pairs

Gears are fundamental components used to transmit power and motion in modern industry. Their health condition monitoring is crucial to ensure reliable operations, prevent unscheduled shutdowns, and minimize human casualties. From this standpoint, the present study proposed a one-dimensional convolutional neural network (1-D CNN) model to diagnose tooth root cracks for standard and asymmetric involute spur gears. A 6-degrees-of-freedom dynamic model of a one-stage spur gear transmission was established to achieve this end and simulate vibration responses of healthy and cracked (25%–50%–75%–100%) standard (20°/20°) and asymmetric (20°/25° and 20°/30°) spur gear pairs. Three levels of signal-to-noise ratios were added to the vibration data to complicate the early fault diagnosis task. The primary consideration of the present study is to investigate the asymmetric gears’ dynamic characteristics and whether tooth asymmetry would yield an advantage in detecting tooth cracks easier to add to the improvements it affords in terms of impact resistance, bending strength, and fatigue life. The findings indicated that the developed 1-D CNN model’s classification accuracy could be improved by up to 12.8% by using an asymmetric (20°/30°) tooth profile instead of a standard (20°/20°) design.

A Two-stream Convolutional Neural Network-based Pornography Recognition Method

The main approach taken to identify pornographic video content is achieved by performing pornography detection on the video content. By extracting features from video key frames and using some common neural network models to recognize the extracted key frame images, a certain accuracy rate can be obtained. However, another key information of video recognition, action information, is ignored, which leads to misclassification of some indistinguishable videos such as sumo wrestling and boxing. A dual-stream convolutional neural network-based pornographic video recognition method is proposed to address this problem. The experimental results show that the dual-stream convolutional neural network effectively improves the recognition rate of indistinguishable pornographic videos.

LANDMARK DETECTION USING DEEP LEARNING

In many computer vision applications, finding landmarks is a crucial challenge. In this thesis, we suggest a unique convolutional neural network-based method for landmark detection (CNNs). On the PASCAL VOC and COCO datasets, our solution surpasses cutting-edge techniques. The landmark detection method we suggest using CNN is built on a three-layer deep network. The input image's features are extracted by the feature extraction layer, which is the first layer. The input image is categorised into one of the pre-established landmark categories in the second layer, which is a classification layer. A localization layer, which is the third layer, locates the landmarks in the input picture. Keywords—CNN,tensorflow_hub,streamlit, geolocation,landmark detection, classification

Large depth-of-field image fusion method for complex gyration class mechanical parts

The local regions of complex gyration class mechanical parts are largely different, which results in the problem that their surface images are partially clear. This problem directly affects the effectiveness of vision methods on detecting surface defects. To tackle this problem, a large depth-of-field (DoF) surface image fusion method is proposed to obtain a full-focus image of complex mechanical parts, such as gyration class mechanical parts. We design an image acquisition platform based on the characteristics of gyration class parts to acquire their surface images accurately and clearly. We then propose a feature detection-based image registration method, by which the registered image can represent the surface information of the part completely and accurately. Additionally, we adopt a convolutional neural network-based image fusion method to achieve a fused large DoF surface image. Experimental studies were conducted to evaluate the performance of the deep-learning-based methods. The experimental results show that the proposed method can completely and clearly fuse the large DoF surface images of complex gyration class mechanical parts. The quality of the fused images has a significant improvement, and the proposed method is significantly more efficient than traditional fusion methods and other deep-learning methods.

Fracture Extraction From Logging Image Using a Dual Encoder-Decoder Architecture With Swin Transformer

Imaging logging is a method of imaging the physical parameters of the borehole wall or the objects around the borehole according to the observation of the geophysical field in the borehole. Imaging logging data can determine the dip angle and structural characteristics of the formation and observe the geometry and development degree of fractures. The performance of existing target segmentation networks relies on large volumes of data. However, logging images are expensive to acquire, so how to effectively extract fractures from small samples of logging images is an urgent problem to be solved. Therefore, we developed a dual encoder-decoder structure using the Swin Transformer, which uses the self-attention mechanism of a hierarchical Vision Transformer with shifted window to model the remote context information. It can overcome the limitations of most convolutional neural network-based methods that cannot establish long-term dependencies and global contextual connections in convolutional operations. In addition, the shifted window mechanism substantially improves the computational efficiency of the model, and the hierarchical structure allows flexibility in modeling at different scales. At the same time, skip connections are established between adjacent layers of the structure, and the higher-level feature maps are stitched with the lower-level feature maps in channel dimensions, which can obtain more high-resolution detail information of fractures, and thus improve the segmentation accuracy. The experimental results show that the performance is better than the mainstream segmentation networks under small training sets of logging images. The effectiveness of our method reveals that it is practical in fracture extraction of logging images.