CNN Framework Research Articles

Self-Supervised Transfer Learning Based on Domain Adaptation for Benign-Malignant Lung Nodule Classification on Thoracic CT.

The spatial heterogeneity is an important indicator of the malignancy of lung nodules in lung cancer diagnosis. Compared with 2D nodule CT images, the 3D volumes with entire nodule objects hold richer discriminative information. However, for deep learning methods driven by massive data, effectively capturing the 3D discriminative features of nodules in limited labeled samples is a challenging task. Different from previous models that proposed transfer learning models in a 2D pattern or learning from scratch 3D models, we develop a self-supervised transfer learning based on domain adaptation (SSTL-DA) 3D CNN framework for benign-malignant lung nodule classification. At first, a data pre-processing strategy termed adaptive slice selection (ASS) is developed to eliminate the redundant noise of the input samples with lung nodules. Then, the self-supervised learning network is constructed to learn robust image representations from CT images. Finally, a transfer learning method based on domain adaptation is designed to obtain discriminant features for classification. The proposed SSTL-DA method has been assessed on the LIDC-IDRI benchmark dataset, and it obtains an accuracy of 91.07% and an AUC of 95.84%. These results demonstrate that the SSTL-DA model achieves quite a competitive classification performance compared with some state-of-the-art approaches.

Crowd Counting by Using Top-k Relations: A Mixed Ground-Truth CNN Framework

Crowd counting has important applications in the environments of smart cities, such as intelligent surveillance. In this paper, we propose a novel convolutional neural network (CNN) framework for crowd counting with mixed ground-truth, called top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation-based network (TKRNet). Specifically, the estimated density maps generated in a coarse-to-fine manner are treated as coarse locations for crowds so as to assist our TKRNet to regress the scattered point-annotated ground truth. Moreover, an adaptive top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relation module (ATRM) is proposed to enhance feature representations by leveraging the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> dependencies between the pixels with an adaptive filtering mechanism. Specifically, we first compute the similarity between two pixels so as to select the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations for each position. Then, a weight normalization operation with an adaptive filtering mechanism is proposed to make the ATRM adaptively eliminate the influence from the low correlation positions in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Finally, a weight attention mechanism is introduced to make the ATRM pay more attention to the positions with high weights in the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> relations. Extensive experimental results demonstrate the effectiveness of our proposed TKRNet on several public datasets in comparison to state-of-the-art methods.

Graph attention autoencoder inspired CNN based brain tumor classification using MRI

Early and accurate detection is a solitary precaution to overcome the brain tumor. Otherwise, it will result in a deadly disease. Brain tumor (BT) detection using magnetic resonance is an essential and challenging job in the medical domain. To combat the aggressive spreading of BT, clinical imaging (MRI and X-Ray) can be an appropriate method for diagnosis. In this paper, we applied attention based image classification, which is a new paradigm for obtaining improved accuracy than the state-of-the-art models. We propose a GATE-CNN (Graph Attention AutoEncoder-Convolution Neural Network) model in this work for the classification of BT. We calculate the attention values of the neighboring pixels on each and every pixel present in the graph then process the graph using GATE framework and the processed graph with attention values is then passed to CNN framework for generation of final output. Further, we apply Adamax optimizer to optimize the training hyperparameters of CNN. We perform the BT classification using the proposed method on three different datasets among benign and malignant BT for first dataset, glioma and pituitary for second dataset, and normal and abnormal BT images for third dataset; all the three datasets consists of MRI images. We then compare the proposed model with a variety of CNN (Deep-CNN, Multi-Input CNN, and basic CNN) models dealing with several types of medical imaging (thermal image, MRI, X-Ray, etc.). Our model results in the accuracy of 98.27% for first dataset, 99.83% for second dataset, and 98.78% for third dataset demonstrating preferable network performance than the already established state-of-the-art CNN models.

Retraction Note to: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

Retraction Note to: Deep CNN framework for retinal disease diagnosis using optical coherence tomography images

A Multiscale CNN Framework for Wireless Technique Classification in Internet of Things

Wireless technique classification (WTC) is of crucial importance in Internet of Things for realizing efficient spectrum sharing and interference management. However, the existing deep-learning-based methods have low classification accuracy, especially at low signal-to-noise ratio levels. In this article, a multiscale convolutional neural network framework is proposed for WTC. A multiscale module is exploited to capture the higher abstraction features. Simulation results demonstrate that our proposed scheme can achieve a better classification performance and a higher convergence speed compared to the state-of-the-art schemes.

A Survey on Face Mask Detection and Social Distancing Measurement Using Machine Learning

Abstract: The purpose of the project “Implementation of Covid-19 Monitoring System using ML-based Face Mask Detectionand Social Distancing Measurement” is to create a tool that identifies the image of a human that can calculate the probability that he/she is wearing a mask or not and another motive of our project is to detect the people and check whether they aremaintaining the social distancingnorms. Using Machine Learning and Object Detection, this study seeks to detect face masksand social separation. The contribution is a strategy for merging more complex classifiers in a "cascade" that allows the image's background areas to be swiftly rejected while more computation is spent on promising object-like regions. While deep learning-based methods for general object detection have progressed dramatically in the previous two years, most approaches to face detection still use the CNN framework, which results in low accuracy and processing performance. We examine the use of MobileNet(pretrained model) and feature extraction in this project, which will yield outstanding results on a variety of object detection benchmarks. YOLO Object detection was used to detect persons in a frame and calculate the distance between them to check for social distancing. As a result, in a real-time situation, this technology tracks persons wearing or not wearing masks and provides social separation by triggering an alert if there is a violation in the scene or in public spaces. This can be combined with current embedded camera infrastructure to allow these analytics, which can be used in a variety of verticals as well as in offices and airport terminals/gates. Keywords: COVID–19, Face Mask, Social Distancing, Machine Learning,MobileNet, YOLOv3, Tensorflow, Keras, OpenCV, Object Detection. Abbreviations: DNN (Deep Neural Network), YOLOv3 (You Only Look Once Version3)

A 3D Tubular Flux Model for Centerline Extraction in Neuron Volumetric Images.

Digital morphology reconstruction from neuron volumetric images is essential for computational neuroscience. The centerline of the axonal and dendritic tree provides an effective shape representation and serves as a basis for further neuron reconstruction. However, it is still a challenge to directly extract the accurate centerline from the complex neuron structure with poor image quality. In this paper, we propose a neuron centerline extraction method based on a 3D tubular flux model via a two-stage CNN framework. In the first stage, a 3D CNN is used to learn the latent neuron structure features, namely flux features, from neuron images. In the second stage, a light-weight U-Net takes the learned flux features as input to extract the centerline with a spatial weighted average strategy to constrain the multi-voxel width response. Specifically, the labels of flux features in the first stage are generated by the 3D tubular model which calculates the geometric representations of the flux between each voxel in the tubular region and the nearest point on the centerline ground truth. Compared with self-learned features by networks, flux features, as a kind of prior knowledge, explicitly take advantage of the contextual distance and direction distribution information around the centerline, which is beneficial for the precise centerline extraction. Experiments on two challenging datasets demonstrate that the proposed method outperforms other state-of-the-art methods by 18% and 35.1% in F1-measurement and average distance scores at the most, and the extracted centerline is helpful to improve the neuron reconstruction performance.

Retinopathy grading with deep learning and wavelet hyper-analytic activations.

Recent developments reveal the prominence of Diabetic Retinopathy (DR) grading. In the past few decades, Wavelet-based DR classification has shown successful impacts and the Deep Learning models, like Convolutional Neural Networks (CNN’s), have evolved in offering the highest prediction accuracy. In this work, the features of the input image are enhanced with the integration of Multi-Resolution Analysis (MRA) and a CNN framework without costing more convolution filters. The bottleneck with conventional activation functions, used in CNN’s, is the nullification of the feature maps that are negative in value. In this work, a novel Hyper-analytic Wavelet (HW) phase activation function is formulated with unique characteristics for the wavelet sub-bands. Instead of dismissal, the function transforms these negative coefficients that correspond to significant edge feature maps. The hyper-analytic wavelet phase forms the imaginary part of the complex activation. And the hyper-parameter of the activation function is selected such that the corresponding magnitude spectrum produces monotonic and effective activations. The performance of 3 CNN models (1 custom, shallow CNN, ResNet with Soft attention, Alex Net for DR) with spatial–Wavelet quilts is better. With the spatial–Wavelet quilts, the Alex Net for DR has an improvement with an 11% of accuracy level (from 87 to 98%). The highest accuracy level of 98% and the highest Sensitivity of 99% are attained through Modified Alex Net for DR. The proposal also illustrates the visualization of the negative edge preservation with assumed image patches. From this study, the researcher infers that models with spatial–Wavelet quilts, with the hyper-analytic activations, have better generalization ability. And the visualization of heat maps provides evidence of better learning of the feature maps from the wavelet sub-bands.

Automated quality assessment of chest radiographs based on deep learning and linear regression cascade algorithms.

Develop and evaluate the performance of deep learning and linear regression cascade algorithms for automated assessment of the image layout and position of chest radiographs. This retrospective study used 10 quantitative indices to capture subjective perceptions of radiologists regarding image layout and position of chest radiographs, including the chest edges, field of view (FOV), clavicles, rotation, scapulae, and symmetry. An automated assessment system was developed using a training dataset consisting of 1025 adult posterior-anterior chest radiographs. The evaluation steps included: (i) use of a CNN framework based on ResNet - 34 to obtain measurement parameters for quantitative indices and (ii) analysis of quantitative indices using a multiple linear regression model to obtain predicted scores for the layout and position of chest radiograph. In the testing dataset (n = 100), the performance of the automated system was evaluated using the intraclass correlation coefficient (ICC), Pearson correlation coefficient (r), mean absolute difference (MAD), and mean absolute percentage error (MAPE). The stepwise regression showed a statistically significant relationship between the 10 quantitative indices and subjective scores (p < 0.05). The deep learning model showed high accuracy in predicting the quantitative indices (ICC = 0.82 to 0.99, r = 0.69 to 0.99, MAD = 0.01 to 2.75). The automatic system provided assessments similar to the mean opinion scores of radiologists regarding image layout (MAPE = 3.05%) and position (MAPE = 5.72%). Ten quantitative indices correlated well with the subjective perceptions of radiologists regarding the image layout and position of chest radiographs. The automated system provided high performance in measuring quantitative indices and assessing image quality. • Objective and reliable assessment for image quality of chest radiographs is important for improving image quality and diagnostic accuracy. • Deep learning can be used for automated measurements of quantitative indices from chest radiographs. • Linear regression can be used for interpretation-based quality assessment of chest radiographs.

Prediction of material properties of propellant waste modified bricks through microstructures by Topographic independent component analysis coupled 3D Convolution neural networks

PurposeDeciphering materials properties from the textural information of microstructural images is still a challenge. ObjectiveProposes a Topographic Independent Component Analysis coupled with 3-D Convolutional Neural Network (TICA- 3DCNN) architecture that processes the features from scanning electron microscope (SEM) images to predict the material properties of high energy propellant (HEP) modified bricks. MethodFirst, high-energy modified bricks with ten different HEP additives fractions is prepared, and their microstructural information is acquired with SEM. Then, SEM images of each group are pre-processed and subjected to the TICA analysis to decipher the distinct higher-order microstructural dependencies, such as co-activation of components. These microstructural co-activation components are subsequently ordered and passed into 3D CNN architecture to map them with the material properties such as water absorption, compressive strength, density, and porosity. An extreme learning machine is employed as a fully connected classification layer in the 3D-CNN architecture. Finally, the performance of the proposed TICA - 3DCNN approach is compared with the traditional 2D CNN frameworks such as Inception-v4 and Faster RCNN. ResultsProposed TICA-3DCNN architecture has performed with higher sensitivity and specificity than Inception-v4 and Faster RCNN architecture in deciphering microstructural textural properties and mapping them with materials properties. ConclusionThe Proposed model can be used in the construction and building materials to predict material properties through microstructural features.

A Proactive Recognition System for Detecting Commercial Vehicle Driver’s Distracted Behavior

Road traffic accidents regarding commercial vehicles have been demonstrated as an important culprit restricting the steady development of the social economy, which are closely related to the distracted behavior of drivers. However, the existing driver’s distracted behavior surveillance systems for monitoring and preventing the distracted behavior of drivers still have some shortcomings such as fewer recognition objects and scenarios. This study aims to provide a more comprehensive methodological framework to demonstrate the significance of enlarging the recognition objects, scenarios and types of the existing driver’s distracted behavior recognition systems. The driver’s posture characteristics were primarily analyzed to provide the basis of the subsequent modeling. Five CNN sub-models were established for different posture categories and to improve the efficiency of recognition, accompanied by a holistic multi-cascaded CNN framework. To suggest the best model, image data sets of commercial vehicle driver postures including 117,410 daytime images and 60,480 night images were trained and tested. The findings demonstrate that compared to the non-cascaded models, both daytime and night cascaded models show better performance. Besides, the night models exhibit worse accuracy and better speed relative to their daytime model counterparts for both non-cascaded and cascaded models. This study could be used to develop countermeasures to improve driver safety and provide helpful information for the design of the driver’s real-time monitoring and warning system as well as the automatic driving system. Future research could be implemented to combine the vehicle state parameters with the driver’s microscopic behavior to establish a more comprehensive proactive surveillance system.

Deep Learning CNN Framework for Detection and Classification of Internet Worms

A worm is a standalone program, which is self-replicating malware that distributes itself to other computers and networks. An Internet worm can spread across the network and infect millions of computers in truly little time and the damages caused from such attacks are considered extremely high. In addition, these worms also affect the network packet and its performance, where the packets are analyzed by the signature-based intrusion detection system (IDS) and the network performance is analyzed by the NetFlow based IDS. Hence, this article proposes a joint detection of both the signature based and NetFlow based Internet worms using deep learning convolution neural network (DLCNN) with respect to various attacks and it can also prevent the suspicious actions of attackers (cyber-criminals). Additionally, it provides the security for users’ data maintenance, countermeasures, and controls the spreading of the internet worms. The effectiveness of proposed DLCNN model is evaluated using both packet capture (PCAP) and KDD-CUP-99 datasets. Finally, various quality metrics are employed to disclose the superiority of proposed DLCNN model as compared existing machine learning, and back propagated neural network models.

Variable Subpixel Convolution Based Arbitrary-Resolution Hyperspectral Pansharpening

Standard hyperspectral (HS) pansharpening rely on fusion to enhance low-resolution HS (LRHS) images to the resolution of their matching panchromatic (PAN) images, whose practical implementation is normally under a stipulation of scale invariance of model across the training phase and the pansharpening phase. By contrast, arbitrary resolution HS (ARHS) pansharpening seeks to pansharpen LRHS images to any user-customized resolutions. For such a new HS pansharpening task, it is not feasible to train and store CNN models for all possible candidate scales, which implies the single model acquired from the training phase should be capable of being generalized to yield HS images with any resolutions in the pansharpening phase. To address the challenge, a novel variable sub-pixel convolution (VSPC)-based CNN (VSPC-CNN) method following our arbitrary upsampling CNN (AU-CNN) framework is developed for ARHS pansharpening. The VSPC-CNN method comprises a two-stage elevating thread. The first stage is to improve the spatial resolution of input HS image to that of the PAN image through a pre-pansharpening module and then a VSPC-encapsulated arbitrary scale attention upsampling (ASAU) module is cascaded for arbitrary resolution adjustment. After training with given scales, it can be generalized to pansharpen HS image to arbitrary scales under the spatial patterns invariance across the training and pansharpening phases. Experimental results from several specific VSPC-CNNs on both simulated and real HS datasets show the superiority of the proposed method.

Deep relational self-Attention networks for scene graph generation

Scene graph generation (SGG) aims to simultaneously detect objects in an image and predict relations for these detected objects. SGG is challenging that requires modeling the contextualized relationships among objects rather than only considering relationships between paired objects. Most existing approaches address this problem by using a CNN or RNN framework, which can not explicitly and effectively models the dense interactions among objects. In this paper, we exploit the attention mechanism and introduce a relational self-attention (RSA) module to simultaneously model the object and relation contexts. By stacking such RSA modules in depth, we obtain a deep relational self-attention network (RSAN), which is able to characterize complex interactions thus facilitating the understanding of object and relation semantics. Extensive experiments on the benchmark Visual Genome dataset demonstrate the effectiveness of RSAN.

Optimisation assisted CNN framework for bearing fault diagnosis

Optimisation assisted CNN framework for bearing fault diagnosis

Optimisation assisted CNN framework for bearing fault diagnosis

Nowadays, intellectual fault diagnosis mechanism with DL schemes was extensively deployed in production firms to develop the effectiveness of fault diagnosis. The rolling bearings connect the support and rotor and are considered as a critical element in rotating equipments. Nevertheless, the working state of bearing varies based on composite operation demand that may drastically corrupt the performances of the intellectual fault diagnosis technique. Thereby, this scheme develops novel fault diagnosis schemes that included the two most important stages like 'feature extraction' and 'classification'. Initially, the features namely, 'empirical wavelet transform', 'empirical mode decomposition' and 'wavelet transform' are extracted. Subsequent to this, the derived features are classified via 'optimised convolutional neural network' is employed. Further, to get better accuracy using adopted model, the weights of CNN is tuned via self-adaptive moth-flame optimisation. Eventually, the primacy of the offered scheme is proven regarding varied measures. Eventually, the proposed technique has obtained a superior value of 0.922, and it is 1.62%, 1.92%, 50.76%, and 57.34%, superior to existing MFO, FF, SVM and RF models for dataset 1.

A CNN Framework With Slow-Fast Band Selection and Feature Fusion Grouping for Hyperspectral Image Change Detection

Change detection approaches can detect changed areas of the same scene at different times. Hyperspectral remote-sensing images contain large amounts of spectral information at high resolution. As hyperspectral datasets become abundant, more and more change detection technologies use hyperspectral images as raw data. Hyperspectral images suffer from band redundancy. There is an urgent need to improve the directionality of change of information features. To solve these problems, in this article, we propose a CNN framework involving slow-fast band selection (SFBS) and feature fusion grouping (SFBS-FFGNET) for hyperspectral image change detection. The main contributions of this article are as follows: 1) based on slow feature analysis (SFA), an SFBS method is proposed, which selects slow and fast feature bands to better extract changed and unchanged features, to more effectively separate changed and unchanged pixels; 2) we used a difference matrix to enrich the level of change information to provide more change characteristics for change detection; and 3) an FFG method was used to generate a more discriminative feature group, and the related loss function was designed. Experimental results on multiple real hyperspectral datasets showed that SFBS can reduce the operating load on the computer and improve the accuracy of change detection, and FFG can also improve the accuracy of change detection. In summary, SFBS-FFGNET is superior to most existing change detection methods.

AI-Enabled Predictive Maintenance Framework for Autonomous Mobile Cleaning Robots.

Vibration is an indicator of performance degradation or operational safety issues of mobile cleaning robots. Therefore, predicting the source of vibration at an early stage will help to avoid functional losses and hazardous operational environments. This work presents an artificial intelligence (AI)-enabled predictive maintenance framework for mobile cleaning robots to identify performance degradation and operational safety issues through vibration signals. A four-layer 1D CNN framework was developed and trained with a vibration signals dataset generated from the in-house developed autonomous steam mopping robot ‘Snail’ with different health conditions and hazardous operational environments. The vibration signals were collected using an IMU sensor and categorized into five classes: normal operational vibration, hazardous terrain induced vibration, collision-induced vibration, loose assembly induced vibration, and structure imbalanced vibration signals. The performance of the trained predictive maintenance framework was evaluated with various real-time field trials with statistical measurement metrics. The experiment results indicate that our proposed predictive maintenance framework has accurately predicted the performance degradation and operational safety issues by analyzing the vibration signal patterns raised from the cleaning robot on different test scenarios. Finally, a predictive maintenance map was generated by fusing the vibration signal class on the cartographer SLAM algorithm-generated 2D environment map.

Mapping Large-Scale Forest Disturbance Types with Multi-Temporal CNN Framework

Forests play a vital role in combating gradual developmental deficiencies and balancing regional ecosystems, yet they are constantly disturbed by man-made or natural events. Therefore, developing a timely and accurate forest disturbance detection strategy is urgently needed. The accuracy of traditional detection algorithms depends on the selection of thresholds or the formulation of complete rules, which inevitably reduces the accuracy and automation level of detection. In this paper, we propose a new multitemporal convolutional network framework (MT-CNN). It is an integrated method that can realize long-term, large-scale forest interference detection and distinguish the types (forest fire and harvest/deforestation) of disturbances without human intervention. Firstly, it uses the sliding window technique to calculate an adaptive threshold to identify potential interference points, and then a multitemporal CNN network is designed to render the disturbance types with various disturbance duration periods. To illustrate the detection accuracy of MT-CNN, we conducted experiments in a large-scale forest area (about 990 km2) on the west coast of the United States (including northwest California and west Oregon) with long time-series Landsat data from 1986 to 2020. Based on the manually annotated labels, the evaluation results show that the overall accuracies of disturbance point detection and disturbance type recognition reach 90%. Also, this method is able to detect multiple disturbances that continuously occurred in the same pixel. Moreover, we found that forest disturbances that caused forest fire repeatedly appear without a significant coupling effect with annual temporal and precipitation variations. Potentially, our method is able to provide large-scale forest disturbance mapping with detailed disturbance information to support forest inventory management and sustainable development.

A Bayesian multiscale CNN framework to predict local stress fields in structures with microscale features

Multiscale computational modelling is challenging due to the high computational cost of direct numerical simulation by finite elements. To address this issue, concurrent multiscale methods use the solution of cheaper macroscale surrogates as boundary conditions to microscale sliding windows. The microscale problems remain a numerically challenging operation both in terms of implementation and cost. In this work we propose to replace the local microscale solution by an Encoder-Decoder Convolutional Neural Network that will generate fine-scale stress corrections to coarse predictions around unresolved microscale features, without prior parametrisation of local microscale problems. We deploy a Bayesian approach providing credible intervals to evaluate the uncertainty of the predictions, which is then used to investigate the merits of a selective learning framework. We will demonstrate the capability of the approach to predict equivalent stress fields in porous structures using linearised and finite strain elasticity theories.