Convolutional Part Research Articles

A segmentation-based algorithm for classification of benign and malignancy Thyroid nodules with multi-feature information.

The aim of this study is to propose a new diagnostic model based on "segmentation + classification" to improve the routine screening of Thyroid nodule ultrasonography by utilizing the key domain knowledge of medical diagnostic tasks. A Multi-scale segmentation network based on a pyramidal pooling structure of multi-parallel void spaces is proposed. First, in the segmentation network, the exact information of the underlying feature space is obtained by an Attention Gate. Second, the inflated convolutional part of Atrous Spatial Pyramid Pooling (ASPP) is cascaded for multiple downsampling. Finally, a three-branch classification network combined with expert knowledge is designed, drawing on doctors' clinical diagnosis experience, to extract features from the original image of the nodule, the regional image of the nodule, and the edge image of the nodule, respectively, and to improve the classification accuracy of the model by utilizing the Coordinate attention (CA) mechanism and cross-level feature fusion. The Multi-scale segmentation network achieves 94.27%, 93.90% and 88.85% of mean precision (mPA), Dice value (Dice) and mean joint intersection (MIoU), respectively, and the accuracy, specificity and sensitivity of the classification network reaches 86.07%, 81.34% and 90.19%, respectively. Comparison tests show that this method outperforms the U-Net, AGU-Net and DeepLab V3+ classical models as well as the nnU-Net, Swin UNetr and MedFormer models that have emerged in recent years. This algorithm, as an auxiliary diagnostic tool, can help physicians more accurately assess the benign or malignant nature of Thyroid nodules. It can provide objective quantitative indicators, reduce the bias of subjective judgment, and improve the consistency and accuracy of diagnosis. Codes and models are available at https://github.com/enheliang/Thyroid-Segmentation-Network.git.

Deep convolutional neural networks for filtering out normal frames in reviewing wireless capsule endoscopy videos

Wireless capsule endoscopy (WCE) has emerged as a valuable non-invasive technique for visualizing the entire gastrointestinal (GI) tract. However, manual evaluation of WCE videos is a time-consuming and costly process. In this study, we present a novel diagnostic assistant system that employs deep convolutional neural networks (DCNNs) to accelerate the evaluation process. Our primary objective is to achieve a high negative predictive value (NPV), which is essential for the efficient identification of normal frames. Six distinct DCNN models were developed and implemented with this objective in mind. The models were trained on a limited dataset encompassing common GI pathologies that reflect real clinical scenarios. Each DCNN architecture comprises a convolutional part derived from renowned pre-trained networks and a custom-designed classifier block optimized for high NPV and classification accuracy. Following a comprehensive assessment utilizing the 5-fold cross-validation approach, the VG_BFCG model was identified as the most effective, exhibiting an average test accuracy of 0.946 and an NPV of 0.983. Moreover, in the event of encountering novel pathologies not present in the training data, our models exhibited robustness in NPV, which is of great importance for practical applications. For example, the DN_BFCG model demonstrated consistent performance, with an NPV exceeding 0.99 across a range of new pathologies. This validates the reliability of our models in clinical settings. Our findings suggest that our developed DCNN architectures have the potential to enhance the efficiency and accuracy of WCE video analysis, which could transform the landscape of gastroenterological diagnostics.

Artificial intelligence analysis applied to facial features for prediction of high-risk coronary atherosclerosis at cardiac CT: the LOMBROSO Trial

Abstract Background Cardiac computed tomography (CT) has recently emerged as an accurate tool for non-invasive evaluation of coronary artery disease (CAD) and coronary atheroscelrosis itself. Nowadays no validated tools are available to predict the presence of high risk atherosclerosis, probably due to the multifactorial pathogenesis of atherosclerosis. Facial features may express both genetic and environmental factors that could be associated to high risk atherosclerosis. Aim of the present study was to verify whether deep learning models applied to facial features may accurately predict the presence of high risk coronary atherosclerosis at cardiac CT. Methods We enrolled a consecutive cohort of 100 patients who underwent clinical indicated cardiac CT for suspected, CAD. Before CT, 10 facial photos were taken from every patients from random fronts views. All cardiac CT were analysed for the presence of non-calcified plaque volume (defined as <150 HU at CT); the non-calcified plaque volume was quantified on a per-patient basis in mm3. We built a deep learning model, exploiting the transfer learning technique; briefly, we implemented an ‘xception’ architecture, joining a pre-trained convolutional part with a specific combination of dense layers, in which an output layer follows a hidden layer with 512 neurons and a dropout layer with a dropout rate=0.2. The batch size, the number of epochs and the learning rate were 16, 20, and 0.0001, respectively. A training set composed of 198 face images was fed into the model, while 80 face images served as test set for the prediction of the presence of elevated volume of non-calcified plaque from patients facial features. Results We present the final results from the first 100 patients enrolled (69 male and 31 female, with mean age of 62±10 years old). In 30 patients cardiac CT resulted completely normal, while in 70 subjects the presence of coronary atherosclerosis was demonstrated. On the test set, we obtained for deep learning facial analysis an AUC equal to 0.79, 1, 0.94, 0.89 for the prediction of presence of high risk plaque, elevated low density plaque volume and elevated non-calcific plaque volume. These AUC data were significatively higher when compared to SCORE2 capability to predict the presence of high risk plaque features, low density plaque volume and non-calcific plaque volume (AUC equal to 0.65, 0.64, 0.66, respectively) among all 100 patients enrolled. Conclusions Prediction of the presence of high risk atherosclerosis from deep learning models applied to facial features appeared to be feasible and promising. Of note, significatively high AUC were demonstrated for deep learning facial analysis when compared to SCORE2. Our results may provide a useful tool for appropriate identification of patients that may merit to underwent cardiac CT, even if asymptomatic, for early identification of high risk atherosclerosis.

Dynamic weight HiLo attention network for medical image multiple organ segmentation

AbstractIn recent years, there has been a surge of research in the field of medical image segmentation using hybrid CNN‐Transformer network architectures. Most of these studies leverage the attention mechanism of ViT to overcome the limitations of CNN architectures in capturing long‐range dependencies. However, these hybrid model approaches also have some potential drawbacks. First, due to the heavy reliance of the Transformer's attention mechanism on global information, it can lead to a significant increase in computational cost when dealing with high‐resolution input images. Furthermore, the convolutional and attention mechanisms in hybrid models have different interpretability in information extraction and decision‐making, which poses a challenge for the interpretability of the convolutional part. Our proposed model, DWHA, addresses these limitations and outperforms state‐of‐the‐art models in a range of medical image segmentation tasks, such as abdominal multiorgan segmentation, automatic cardiac diagnosis, neurostructure segmentation, and skin lesion segmentation, achieving significantly superior performance. Specifically, on the abdominal multiorgan segmentation dataset, DWHA outperformed the previous state‐of‐the‐art baseline by 0.57%; on the neurostructure segmentation dataset, it achieved an improvement of 1.17%; and on the skin lesion segmentation dataset, it achieved an improvement of 0.91%. These significant improvements suggest that DWHA may become the preferred model in the field of medical image segmentation.

CNN and Transformer Fusion for Remote Sensing Image Semantic Segmentation

Semantic segmentation of remote sensing images has been widely used in environmental protection, geological disaster discovery, and natural resource assessment. With the rapid development of deep learning, convolutional neural networks (CNNs) have dominated semantic segmentation, relying on their powerful local information extraction capabilities. Due to the locality of convolution operation, it can be challenging to obtain global context information directly. However, Transformer has excellent potential in global information modeling. This paper proposes a new hybrid convolutional and Transformer semantic segmentation model called CTFuse, which uses a multi-scale convolutional attention module in the convolutional part. CTFuse is a serial structure composed of a CNN and a Transformer. It first uses convolution to extract small-size target information and then uses Transformer to embed large-size ground target information. Subsequently, we propose a spatial and channel attention module in convolution to enhance the representation ability for global information and local features. In addition, we also propose a spatial and channel attention module in Transformer to improve the ability to capture detailed information. Finally, compared to other models used in the experiments, our CTFuse achieves state-of-the-art results on the International Society of Photogrammetry and Remote Sensing (ISPRS) Vaihingen and ISPRS Potsdam datasets.

Predicting Activities of Daily Living for the Coming Time Period in Smart Homes

Activity prediction aims to predict what activities will occur in the future. In smart homes, to facilitate the daily living of the residents, automated or assistive services are provided. To provide these services, the ability of activity prediction is necessary. When we make a prediction, most of the existing works focus on predicting information about the next activity. However, in a smart home environment, compared with just predicting information about the next activity, another type of activity prediction problem has more practical value: predicting what activities will occur in the coming time period of a certain length. The necessity of this type of prediction is due to the purpose of the smart homes and the character of the activities. Many activities in smart homes need preparation time before being performed. Through this type of prediction, activities could be predicted sufficient time before being performed, and there will be adequate time for the smart home system to prepare corresponding automated or assistive services. As more than one activity could occur within the time period in which the prediction is made, this problem is a multilabel classification problem. In this article, we first give a formulation of the problem. Then, we propose a deep learning model to address it. The proposed model consists of the convolutional part, the long short-term memory layer, and the multilabel output layer. Experiments on real-world datasets show the effectiveness of our model.

Deep Learning Based Efficient Pneumonia Detection in Chest X-Ray Images

Medical motion plays an essential role in clinical treatment. The deep neural network is an emerging machine learning method that has proven its potential for different classification tasks. Notably, the convolutional neural network dominates with the best results on varying image classification tasks. However, medical image datasets are hard to collect because it needs a lot of professional expertise to label them. Although it can be detected and treated with very less sophisticated instruments and medication. Therefore, this paper how to apply the Convolutional Neural Network (CNN) based algorithm on a chest X-ray dataset to classify pneumonia. The objective and automated detection of pneumonia represents a serious challenge in medical imaging because the signs of the illness are not obvious in CT or X-ray scans. Further on, it is also an important task, since millions of people die of pneumonia every year. Deep learning-based methods have shown good generalization traits over various problem domains, which prompts researchers around the globe to work tirelessly and come up with more efficient and effective models than earlier. However, this robust nature comes at the cost of high computational resources and, in general, it requires a huge amount of data to train the model efficiently. The latter requirement sometimes cannot be fulfilled, especially in the biomedical field. The main goal is to propose a solution for the above-mentioned problem, using a novel deep neural network architecture. The proposed novelty consists of the use of dropout in the convolutional part of the network. The proposed method was trained and tested on a set of labeled images. This project aims to introduce a deep learning technology based on the computational neural network, which can realize automatic diagnosis of patients with pneumonia in X-ray images.

A New Approach to Descriptors Generation for Image Retrieval by Analyzing Activations of Deep Neural Network Layers.

In this brief, we consider the problem of descriptors construction for the task of content-based image retrieval using deep neural networks. The idea of neural codes, based on fully connected layers' activations, is extended by incorporating the information contained in convolutional layers. It is known that the total number of neurons in the convolutional part of the network is large and the majority of them have little influence on the final classification decision. Therefore, in this brief, we propose a novel algorithm that allows us to extract the most significant neuron activations and utilize this information to construct effective descriptors. The descriptors consisting of values taken from both the fully connected and convolutional layers perfectly represent the whole image content. The images retrieved using these descriptors match semantically very well to the query image, and also, they are similar in other secondary image characteristics, such as background, textures, or color distribution. These features of the proposed descriptors are verified experimentally based on the IMAGENET1M dataset using the VGG16 neural network. For comparison, we also test the proposed approach on the ResNet50 network.

Deep learning-based DOA estimation using CRNN for underwater acoustic arrays

In the marine environment, estimating the direction of arrival (DOA) is challenging because of the multipath signals and low signal-to-noise ratio (SNR). In this paper, we propose a convolutional recurrent neural network (CRNN)-based method for underwater DOA estimation using an acoustic array. The proposed CRNN takes the phase component of the short-time Fourier transform of the array signals as the input feature. The convolutional part of the CRNN extracts high-level features, while the recurrent component captures the temporal dependencies of the features. Moreover, we introduce a residual connection to further improve the performance of DOA estimation. We train the CRNN with multipath signals generated by the BELLHOP model and a uniform line array. Experimental results show that the proposed CRNN yields high-accuracy DOA estimation at different SNR levels, significantly outperforming existing methods. The proposed CRNN also exhibits a relatively short processing time for DOA estimation, extending its applicability.

Deep learning applied to facial features for prediction of high risk coronary atherosclerosis at cardiac CT: preliminary data from LOMBROSO Trial

Abstract Background Cardiac computed tomography (CT) emerged as an accurate tool for non-invasive evaluation of coronary artery disease (CAD), being able to identify high risk non-calcified atherosclerosis. Identification of high risk CAD in its asymptomatic stage could be an interest target for medical therapy. Nowadays no validated tools are available to predict the presence of high risk atherosclerosis, probably due to the multifactorial pathogenesis of atherosclerosis. Facial features may express both genetic and environmental factors that could be associated to high risk atherosclerosis. Aim of the present study was to verify whether deep learning models applied to facial features may accurately predict the presence of high risk coronary atherosclerosis evaluated at cardiac CT Methods We enrolled a consecutive cohort of patients who underwent clinical indicated cardiac CT for suspected, CAD. Before CT, 10 facial photos were taken from every patients from random fronts views. All cardiac CT were analysed for the presence of non-calcified plaque volume (defined as <150 HU at CT); the non-calcified plaque volume was quantified on a per-patient basis in mm3 and a cut off of >23 mm3 was used to define a patients with an elevated volume non-calcified plaque We built a deep learning model, exploiting the transfer learning technique; briefly, we implemented an “xception” architecture, joining a pre-trained convolutional part with a specific combination of dense layers, in which an output layer follows a hidden layer with 512 neurons and a dropout layer with a dropout rate=0.2. The batch size, the number of epochs and the learning rate were 16, 20, and 0.0001, respectively. A training set composed of 198 face images was fed into the model, while 20 face images served as test set for the prediction of the presence of elevated volume of non-calcified plaque from patients facial features. Results We present early results from the first 20 patients enrolled (12 male and 8 female, with mean age of 73±13 years old). In 9 patients cardiac CT resulted completely normal, while in 11 subjects the presence of coronary atherosclerosis was demonstrated. Among them, 9 patients presented non-calcified coronary atherosclerosis, while 6 had an elevated volume of non-calcified plaque. On the test set, we obtained an accuracy, sensitivity, specificity, positive predictive value, negative predictive values and and AUC equal to 0.90, 1, 0.8, 0.83, 1, and 0.99, respectively for the prediction of the presence of an elevated volume of non-calcified plaque from facial features among all 20 patients enrolled. Conclusions Prediction of the presence of high risk atherosclerosis from deep learning models applied to facial features appeared to be feasible and promising. Our results may provide a useful tool for appropriate identification of patients that may merit to underwent cardiac CT, even if asymptomatic, for early identification of high risk atherosclerosis Funding Acknowledgement Type of funding sources: None.

Facial expression recognition via a jointly-learned dual-branch network

Human emotion recognition depends on facial expressions, and essentially on the extraction of relevant features. Accurate feature extraction is generally difficult due to the influence of external interference factors and the mislabelling of some datasets, such as the Fer2013 dataset. Deep learning approaches permit an automatic and intelligent feature extraction based on the input database. But, in the case of poor database distribution or insufficient diversity of database samples, extracted features will be negatively affected. Furthermore, one of the main challenges for efficient facial feature extraction and accurate facial expression recognition is the facial expression datasets, which are usually considerably small compared to other image datasets. To solve these problems, this paper proposes a new approach based on a dual-branch convolutional neural network for facial expression recognition, which is formed by three modules: The two first ones ensure features engineering stage by two branches, and features fusion and classification are performed by the third one. In the first branch, an improved convolutional part of the VGG network is used to benefit from its known robustness, the transfer learning technique with the EfficientNet network is applied in the second branch, to improve the quality of limited training samples in datasets. Finally, and in order to improve the recognition performance, a classification decision will be made based on the fusion of both branches’ feature maps. Based on the experimental results obtained on the Fer2013 and CK+ datasets, the proposed approach shows its superiority compared to several state-of-the-art results as well as using one model at a time. Those results are very competitive, especially for the CK+ dataset, for which the proposed dual branch model reaches an accuracy of 99.32, while for the FER-2013 dataset, the VGG-inspired CNN obtains an accuracy of 67.70, which is considered an acceptable accuracy, given the difficulty of the images of this dataset.

Detection of pneumonia using convolutional neural networks and deep learning

The objective and automated detection of pneumonia represents a serious challenge in medical imaging, because the signs of the illness are not obvious in CT or X-ray scans. Further on, it is also an important task, since millions of people die of pneumonia every year. The main goal of this paper is to propose a solution for the above mentioned problem, using a novel deep neural network architecture. The proposed novelty consists in the use of dropout in the convolutional part of the network. The proposed method was trained and tested on a set of 5856 labeled images available at one of Kaggle’s many medical imaging challenges. The chest X-ray images (anterior-posterior) were selected from retrospective cohorts of pediatric patients, aged between one and five years, from Guangzhou Women and Children’s Medical Center, Guangzhou, China. Results achieved by our network would have placed first in the Kaggle competition with the following metrics: 97.2% accuracy, 97.3% recall, 97.4% precision and AUC=0.982, and they are competitive with current state-of-the-art solutions.

Harnessing feature extraction capacities from a pre-trained convolutional neural network (VGG-16) for the unsupervised distinction of aortic outflow velocity profiles in patients with severe aortic stenosis.

Hypothesizing that aortic outflow velocity profiles contain more valuable information about aortic valve obstruction and left ventricular contractility than can be captured by the human eye, features of the complex geometry of Doppler tracings from patients with severe aortic stenosis (AS) were extracted by a convolutional neural network (CNN). After pre-training a CNN (VGG-16) on a large data set (ImageNet data set; 14 million images belonging to 1000 classes), the convolutional part was employed to transform Doppler tracings to 1D arrays. Among 366 eligible patients [age: 79.8 ± 6.77 years; 146 (39.9%) women] with pre-procedural echocardiography and right heart catheterization prior to transcatheter aortic valve replacement (TAVR), good quality Doppler tracings from 101 patients were analysed. The convolutional part of the pre-trained VGG-16 model in conjunction with principal component analysis and k-means clustering distinguished two shapes of aortic outflow velocity profiles. Kaplan-Meier analysis revealed that mortality in patients from Cluster 2 (n = 40, 39.6%) was significantly increased [hazard ratio (HR) for 2-year mortality: 3; 95% confidence interval (CI): 1-8.9]. Apart from reduced cardiac output and mean aortic valve gradient, patients from Cluster 2 were also characterized by signs of pulmonary hypertension, impaired right ventricular function, and right atrial enlargement. After training an extreme gradient boosting algorithm on these 101 patients, validation on the remaining 265 patients confirmed that patients assigned to Cluster 2 show increased mortality (HR for 2-year mortality: 2.6; 95% CI: 1.4-5.1, P-value: 0.004). Transfer learning enables sophisticated pattern recognition even in clinical data sets of limited size. Importantly, it is the left ventricular compensation capacity in the face of increased afterload, and not so much the actual obstruction of the aortic valve, that determines fate after TAVR.

Abstract 9220: Employing a Convolutional Neural Network Facilitates Unsupervised Clustering of Aortic Outflow Velocity Profiles from Patients With Severe Aortic Stenosis

Introduction: The aortic outflow velocity profile is key to diagnose severe aortic stenosis (AS), and its shape reflects both aortic valve obstruction and left ventricular contractility. Hypothesis: Hypothesizing that aortic outflow velocity profiles contain more valuable information than can be captured by the human eye, features of the complex geometry of Doppler tracings were extracted by a convolutional neural network (CNN) pre-trained on big data (transfer learning). Methods: After pre-training a CNN (VGG-16) on a large data set (ImageNet data set; 14 million images belonging to 1,000 classes), the convolutional part was employed to transform Doppler tracings to 1D arrays. Principal component analysis (PCA) and k-means clustering of those 1D arrays were performed, and association between cluster and 2-year all-cause mortality was hereinafter assessed. Among 366 eligible patients with pre-procedural echocardiography and right heart catheterization prior to transcatheter aortic valve replacement (TAVR) for severe AS, good quality Doppler tracings from 101 patients were analyzed. Results: Among 101 patients (mean age: 79.3 ± 6.78 years; 49 [48.5%] women), estimated 2-year survival was 83.0% (95% CI: 75.1-91.7%). The convolutional part of the pre-trained VGG-16 model in conjunction with PCA and k-means clustering distinguished 2 shapes of aortic outflow velocity profiles. Kaplan-Meier analysis revealed that mortality in patients from cluster 2 (n = 40 [39.6%]) was significantly increased (HR for 2-year mortality: 3 [95% CI: 1-8.9]). Apart from reduced cardiac output and mean aortic valve gradient (AVG mean ), patients from cluster 2 were also characterized by signs of pulmonary hypertension, impaired right ventricular function and right atrial enlargement. Conventional dichotomization according to AVG mean confirmed that patients with AVG mean < 40 mmHg (n = 54 [53.5%]) decease earlier, but no statistical significance was reached (HR for 2-year mortality: 1.8 [95% CI: 0.59-5.2]). Conclusions: This is a proof-of-principle that transfer learning aids in agnostic interpretation of aortic outflow velocity profiles, thereby holding the promise to refine prognostic resolution for survival after TAVR.

Emulation of wildland fire spread simulation using deep learning

Numerical simulation of wildland fire spread is useful to predict the locations that are likely to burn and to support decision in an operational context, notably for crisis situations and long-term planning. For short-term, the computational time of traditional simulators is too high to be tractable over large zones like a country or part of a country, especially for fire danger mapping.This issue is tackled by emulating the area of the burned surface returned after simulation of a fire igniting anywhere in Corsica island and spreading freely during one hour, with a wide range of possible environmental input conditions. A deep neural network with a hybrid architecture is used to account for two types of inputs: the spatial fields describing the surrounding landscape and the remaining scalar inputs.After training on a large simulation dataset, the network shows a satisfactory approximation error on a complementary test dataset with a MAPE of 32.8%. The convolutional part is pre-computed and the emulator is defined as the remaining part of the network, saving significant computational time. On a 32-core machine, the emulator has a speed-up factor of several thousands compared to the simulator and the overall relationship between its inputs and output is consistent with the expected physical behavior of fire spread. This reduction in computational time allows the computation of one-hour burned area map for the whole island of Corsica in less than a minute, opening new application in short-term fire danger mapping.

Local Features Based Deep Learning for Mammographic Image Classification: In Comparison to CNN Models

Abstract Convolutional neural networks (CNN) have achieved tremendous success in tackling various tasks of computer vision problems such as classification of images and recognition of objects. The general trend has been to make networks larger and more complex in order to achieve higher precision. In any case, these advances to improve precision are not really making the systems increasingly effective for size and speed. In this report, an attempt to overcome the aforementioned issues is described. Some well-known scale and rotation invariant local features such as SIFT, SURF, ORB, BRISK, and KAZE were first observed to check which of these having the strong potentiality to replace inclusively the convolutional layers of CNN models. To do this, while retaining the fully connected layers of the corresponding CNN model as the classifier, the convolutional parts of the model were inclusively taken out and assigned the high-level feature maps at the flatten layer with scale and rotation invariant local features. The computational experiments were carried out using the challenging mammographic images provided by MIAS (mammographic image analysis society). It contains 322 grayscale images from 2 different severity, each of which has 1024x1024 pixels. Employing only the fully connected layers of the VGG16 model as the classifier, it is found that SIFT and SURF based features offer much better training and testing accuracies compared to ORB, BRISK, and KAZE in classifying MIAS image datasets. Using SIFT and SURF local features and employing only VGG16 or MobileNetV2 fully connected layers as a classifier, it can be shown that SIFT and SURF based deep learning exhibit state-of-the-art performance on MIAS datasets. SIFT and SURF based deep learning are capable of outperforming the performance of the original VGG16 and MobileNetV2 configuration in terms of accuracy and training time needed on the same datasets.

Skeleton-Based Emotion Recognition Based on Two-Stream Self-Attention Enhanced Spatial-Temporal Graph Convolutional Network.

Emotion recognition has drawn consistent attention from researchers recently. Although gesture modality plays an important role in expressing emotion, it is seldom considered in the field of emotion recognition. A key reason is the scarcity of labeled data containing 3D skeleton data. Some studies in action recognition have applied graph-based neural networks to explicitly model the spatial connection between joints. However, this method has not been considered in the field of gesture-based emotion recognition, so far. In this work, we applied a pose estimation based method to extract 3D skeleton coordinates for IEMOCAP database. We propose a self-attention enhanced spatial temporal graph convolutional network for skeleton-based emotion recognition, in which the spatial convolutional part models the skeletal structure of the body as a static graph, and the self-attention part dynamically constructs more connections between the joints and provides supplementary information. Our experiment demonstrates that the proposed model significantly outperforms other models and that the features of the extracted skeleton data improve the performance of multimodal emotion recognition.

A novel complex network-based deep learning method for characterizing gas\u2013liquid two-phase flow

Gas–liquid two-phase flow widely exits in production and transportation of petroleum industry. Characterizing gas–liquid flow and measuring flow parameters represent challenges of great importance, which contribute to the recognition of flow regime and the optimal design of industrial equipment. In this paper, we propose a novel complex network-based deep learning method for characterizing gas–liquid flow. Firstly, we map the multichannel measurements to multiple limited penetrable visibility graphs (LPVGs) and obtain their degree sequences as the graph representation. Based on the degree distribution, we analyze the complicated flow behavior under different flow structures. Then, we design a dual-input convolutional neural network to fuse the raw signals and the graph representation of LPVGs for the classification of flow structures and measurement of gas void fraction. We implement the model with two parallel branches with the same structure, each corresponding to one input. Each branch consists of a channel-projection convolutional part, a spatial–temporal convolutional part, a dense block and an attention module. The outputs of the two branches are concatenated and fed into several full connected layers for the classification and measurement. At last, our method achieves an accuracy of 95.3% for the classification of flow structures, and a mean squared error of 0.0038 and a mean absolute percent error of 6.3% for the measurement of gas void fraction. Our method provides a promising solution for characterizing gas–liquid flow and measuring flow parameters.

NLDN: Non-local dehazing network for dense haze removal

Single image dehazing is one of the most challenging and important tasks in computer vision and image processing. In this paper, we propose a Non-local Dehazing Network (NLDN), which learns the mapping between hazy images and haze-free images. Our network architecture consists three components: the first is full point-wise convolutional part, which extracts Non-local statistical regularities; the second is feature combination part, which learns the spatial relation of statistical regularities; the third is reconstruction part, which recovers the haze-free image by the features extracted from the second part. By using these three components, we obtain a high quality dehazing result. Experimental results show that our method performs favorably against other state-of-the-art methods on both synthetic dataset and real-world images.

Crowd Counting Network with Self-attention Distillation

Context information is essential for crowd counting network to estimate crowd numbers, especially in the congested scene accurately. However, shallow layers of common crowd counting networks (i.e., congested scene recognition network) do not own large receptive filed so that they can’t efficiently utilize context information from the crowd scene. To solve this problem, in this paper, we propose a crowd counting network with self-attention distillation. Each input image is first sent to the visual geometry group (VGG)-16 network for feature extracting. Then, the extracted features are processed by the dilated convolutional part for the final crowd density estimation. Specially, we apply self-attention distillation strategy at different locations of the dilated convolutional part to use the global context information from the deeper layers to guide the shallower layers to learn. We compare our method with the other state-of-the-art works on the UCF-QNRF dataset, and the experiment results demonstrate the superiority of our method.