Pre-trained Residual Network Research Articles

Artificial intelligence-estimated electrocardiographic age as a recurrence predictor after atrial fibrillation catheter ablations

Abstract Aims The application of artificial intelligence (AI) algorithms to 12-lead electrocardiogram (ECG) provides promising age prediction models. We explored whether the gap between the pre-procedural AI-ECG age and chronological age can predict atrial fibrillation (AF) recurrence after catheter ablation. Methods We validated a pre-trained residual network (ResNet)-based model for age prediction on four independent multinational datasets: CODE-15%, PTB-XL, UK Biobank, and SaMi-Trop cohorts. Then, we evaluated AI-estimated ECG (AI-ECG) age for AF recurrence in a single-center AF catheter ablation (AFCA) cohort, which included 4,794 patients with de-novo AFCA and pre-procedural sinus rhythm ECGs. We categorized the AI-ECG age gap into two groups: aged-ECG (>13 year) and normal age (≤13 year) groups based on the maximum log-likelihood for AF recurrence according to the difference between chronological age and AI-ECG age. Results The ResNet-based AI-ECG age model successfully reproduced the chronological age on the independent datasets (total ECG number=414,804): CODE-15% (r=0.83), PTB-XL (r=0.74), UK Biobank (r=0.53), and SaMi-Trop (r=0.60). During median 22 (9-47) months after AFCA, patients with aged-ECG had a significantly higher AF recurrence rate (adjusted hazard ratio 1.31, 95% confidence interval [1.17-1.48], p<0.001) than normal age group. In the sub-group analyses, the aged-ECG affected more in patients with paroxysmal AF than in non-paroxysmal AF (P for interactions <0.001) and after cryoballoon pulmonary vein isolation (Cryo-PVI) than radiofrequency PVI (P for interactions <0.001). Conclusions Pre-procedural AI- ECG age has a prognostic value for AF recurrence after AF catheter ablation.

Fabric surface defect classification and systematic analysis using a cuckoo search optimized deep residual network

Fabric defects can significantly impact the quality of a textile product. By analyzing the types and frequencies of defects, manufacturers can identify process inefficiencies, equipment malfunctions, or operator errors. Although deep learning networks are accurate in classification applications, some defects may be subtle and difficult to detect, while others may have complex patterns or occlusions. CNNs may struggle to capture a wide range of defect variations and generalize well to unseen defects. Discriminating between genuine defects and benign variations requires sophisticated feature extraction and modeling techniques. This paper proposes a residual network-based CNN model to enhance the classification of fabric defects. A pretrained residual network, ResNet50, is fine-tuned to classify fabric defects into four categories: holes, objects, oil spots, and thread errors on the fabric surface. The fine-tuned network is further optimized via cuckoo search optimization using classification error as a fitness function. The network is systematically analyzed at different layers, and the investigation of classification results are reported using a confusion matrix and classification accuracy for each class. The experimental results confirm that the proposed model achieved superior performance with 95.36% accuracy and a 95.35% F1 score for multiclass classification. In addition, the proposed model achieved higher accuracy with similar or fewer trainable parameters than traditional deep CNN networks.

Small Sample Fiber Full State Diagnosis Based on Fuzzy Clustering and Improved ResNet Network

The optical time domain reflectometer (OTDR) curve features of communication fibers exhibit subtle differences among their normal, subhealthy, and faulty operating states, making it challenging for existing machine learning-based fault diagnosis algorithms to extract these minute features. In addition, the OTDR curve field fault data are scarce, and data-driven deep neural network that needs a lot of data training cannot meet the requirements. In response to this issue, this paper proposes a communication fiber state diagnosis model based on fuzzy clustering and an improved ResNet. First, the pretrained residual network (ResNet) is modified by removing the classification layer and retaining the feature extraction layers. A global average pooling (GAP) layer is designed as a replacement for the fully connected layer. Second, fuzzy clustering, instead of the softmax classification layer, is employed in ResNet for its characteristic of requiring no subsequent data training. The improved model requires only a small amount of sample training to optimize the parameters of the GAP layer, thereby accommodating state diagnosis in scenarios with limited data availability. During the diagnosis process, the OTDR curves are input into the network, resulting in 512 features outputted in the GAP layer. These features are used to construct a feature vector matrix, and a dynamic clustering graph is formed using fuzzy clustering to realize the fiber state diagnosis. Through on-site data detection and validation, it has been demonstrated that the improved ResNet can effectively identify the full cycle of fiber states.

Transfer learning for versatile and training free high content screening analyses

High content screening (HCS) is a technology that automates cell biology experiments at large scale. A High Content Screen produces a high amount of microscopy images of cells under many conditions and requires that a dedicated image and data analysis workflow be designed for each assay to select hits. This heavy data analytic step remains challenging and has been recognized as one of the burdens hindering the adoption of HCS. In this work we propose a solution to hit selection by using transfer learning without additional training. A pretrained residual network is employed to encode each image of a screen into a discriminant representation. The deep features obtained are then corrected to account for well plate bias and misalignment. We then propose two training-free pipelines dedicated to the two main categories of HCS for compound selection: with or without positive control. When a positive control is available, it is used alongside the negative control to compute a linear discriminant axis, thus building a classifier without training. Once all samples are projected onto this axis, the conditions that best reproduce the positive control can be selected. When no positive control is available, the Mahalanobis distance is computed from each sample to the negative control distribution. The latter provides a metric to identify the conditions that alter the negative control’s cell phenotype. This metric is subsequently used to categorize hits through a clustering step. Given the lack of available ground truth in HCS, we provide a qualitative comparison of the results obtained using this approach with results obtained with handcrafted image analysis features for compounds and siRNA screens with or without control. Our results suggests that the fully automated and generic pipeline we propose offers a good alternative to handcrafted dedicated image analysis approaches. Furthermore, we demonstrate that this solution select conditions of interest that had not been identified using the primary dedicated analysis. Altogether, this approach provides a fully automated, reproducible, versatile and comprehensive alternative analysis solution for HCS encompassing compound-based or downregulation screens, with or without positive controls, without the need for training or cell detection, or the development of a dedicated image analysis workflow.

DCCAFN: deep convolution cascade attention fusion network based on imaging genomics for prediction survival analysis of lung cancer

Recently, lung cancer prediction based on imaging genomics has attracted great attention. However, such studies often have many challenges, such as small sample size, high-dimensional information redundancy, and the inefficiency of multimodal fusion. Therefore, in this paper, a deep convolution cascade attention fusion network (DCCAFN) based on imaging genomics is proposed for the prediction of lung cancer patients’ survival. The network consists of three modules: an image feature extraction module (IFEM), a gene feature extraction module (GFEM), and an attention fusion network (AFN). In the IFEM, a pretrained residual network based on transfer learning is used to extract deep image features to fully capture the computed tomography (CT) image information conducive to prognosis prediction. In the GFEM, the F-test is first used for gene screening to eliminate redundant information, and then, a cascade network with the convolution cascade module (CCM) that contains a convolution operation, a pooling operation, and an ensemble forest classifier is designed to better extract the gene features. In the AFN, a bimodal attention fusion mechanism is proposed to fuse deep image features and gene features to improve the performance of predicting lung cancer survival. The experimental results show that the DCCAFN model achieves good performance, and its accuracy and AUC are 0.831 and 0.816, respectively. It indicates that the model is an effective multimodal data fusion method for predicting the survival prognosis of lung cancer, which can greatly help physicians stratify patients' risks, and achieve personalized treatment for improving the quality of patients' lives.

Computer-Aided Diagnosis for Tuberculosis Classification with Water Strider Optimization Algorithm

Computer-aided diagnosis (CAD) models exploit artificial intelligence (AI) for chest X-ray (CXR) examination to identify the presence of tuberculosis (TB) and can improve the feasibility and performance of CXR for TB screening and triage. At the same time, CXR interpretation is a time-consuming and subjective process. Furthermore, high resemblance among the radiological patterns of TB and other lung diseases can result in misdiagnosis. Therefore, computer-aided diagnosis (CAD) models using machine learning (ML) and deep learning (DL) can be designed for screening TB accurately. With this motivation, this article develops a Water Strider Optimization with Deep Transfer Learning Enabled Tuberculosis Classification (WSODTL-TBC) model on Chest X-rays (CXR). The presented WSODTL-TBC model aims to detect and classify TB on CXR images. Primarily, the WSODTL-TBC model undergoes image filtering techniques to discard the noise content and U-Net-based image segmentation. Besides, a pre-trained residual network with a two-dimensional convolutional neural network (2D-CNN) model is applied to extract feature vectors. In addition, the WSO algorithm with long short-term memory (LSTM) model was employed for identifying and classifying TB, where the WSO algorithm is applied as a hyperparameter optimizer of the LSTM methodology, showing the novelty of the work. The performance validation of the presented WSODTL-TBC model is carried out on the benchmark dataset, and the outcomes were investigated in many aspects. The experimental development pointed out the betterment of the WSODTL-TBC model over existing algorithms.

An Automatic Premature Ventricular Contraction Recognition System Based on Imbalanced Dataset and Pre-Trained Residual Network Using Transfer Learning on ECG Signal

The development of automatic monitoring and diagnosis systems for cardiac patients over the internet has been facilitated by recent advancements in wearable sensor devices from electrocardiographs (ECGs), which need the use of patient-specific approaches. Premature ventricular contraction (PVC) is a common chronic cardiovascular disease that can cause conditions that are potentially fatal. Therefore, for the diagnosis of likely heart failure, precise PVC detection from ECGs is crucial. In the clinical settings, cardiologists typically employ long-term ECGs as a tool to identify PVCs, where a cardiologist must put in a lot of time and effort to appropriately assess the long-term ECGs which is time consuming and cumbersome. By addressing these issues, we have investigated a deep learning method with a pre-trained deep residual network, ResNet-18, to identify PVCs automatically using transfer learning mechanism. Herein, features are extracted by the inner layers of the network automatically compared to hand-crafted feature extraction methods. Transfer learning mechanism handles the difficulties of required large volume of training data for a deep model. The pre-trained model is evaluated on the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) Arrhythmia and Institute of Cardiological Technics (INCART) datasets. First, we used the Pan-Tompkins algorithm to segment 44,103 normal and 6423 PVC beats, as well as 106,239 normal and 9987 PVC beats from the MIT-BIH Arrhythmia and IN-CART datasets, respectively. The pre-trained model employed the segmented beats as input after being converted into 2D (two-dimensional) images. The method is optimized with the using of weighted random samples, on-the-fly augmentation, Adam optimizer, and call back feature. The results from the proposed method demonstrate the satisfactory findings without the using of any complex pre-processing and feature extraction technique as well as design complexity of model. Using LOSOCV (leave one subject out cross-validation), the received accuracies on MIT-BIH and INCART are 99.93% and 99.77%, respectively, suppressing the state-of-the-art methods for PVC recognition on unseen data. This demonstrates the efficacy and generalizability of the proposed method on the imbalanced datasets. Due to the absence of device-specific (patient-specific) information at the evaluating stage on the target datasets in this study, the method might be used as a general approach to handle the situations in which ECG signals are obtained from different patients utilizing a variety of smart sensor devices.

Using XAI for Deep Learning-Based Image Manipulation Detection with Shapley Additive Explanation

In the arena of image forensics, detecting manipulations in an image is extremely significant because of the use of images in different fields. Various detection techniques have been suggested in the literature that are based on digging out the features from images to unveil the traces left by manipulation operations. In this paper, a deep learning-based approach is proposed in which a residual network is used to learn deep, complex features from preprocessed images for classification into authentic and forged images. There is statistical symmetry in similar types of images and asymmetry in different types of images. The proposed scheme can highlight the statistical asymmetry between authentic and forged images. In the proposed scheme, firstly, an RGB image is analyzed for different JPEG compression levels. The obtained difference between the error levels is used to extract enhanced LBP code. Then, the scale- and direction-invariant LBP (SD-LBP) code is transformed into SD-LBP feature maps to feed to a deep residual network. Next, the concept of explainable artificial intelligence (XAI) is used to help provide explanations and interpret the output, thereby raising the credibility of the proposed approach. The unique feature selection approach employed is the kernel SHAP method, which is focused on the Shapley values. This technique is used to pinpoint the specific characteristics that are responsible for the aberrant behavior of the forged images dataset. Later, the deep learning-based model is trained and validated using these feature sets. A pre-activation version of ResNet-50 architecture is used that achieved an accuracy of 99.31%, 99.52%, 98.05%, and 99.10% on CASIA v1, CASIA v2, IMD 2020, and DVMM datasets, respectively. The capability of the pretrained residual network and rich textural features, which are scale- and direction-invariant, helps to expand the detection accuracy of the proposed approach. The results confirmed that the method either produced competitive results or outperformed existing methods.

Automated accurate fire detection system using ensemble pretrained residual network

Nowadays, fires have been commonly seen worldwide and especially forest fires are big disasters for humanity. The prime objective of this work is to develop an accurate fire warning model by using images. In this work, two new deep feature engineering models are proposed to detect the fire accurately using images. To create deep features, residual networks (ResNet) are chosen since these networks are one of the highly accurate convolutional neural networks. In this work, four pretrained ResNets: ResNet18, ResNet50, ResNet101, and InceptionResNetV2 are used. These networks were trained using a cluster of ImageNet dataset and features were extracted using the last pooling and fully connected layers of these networks. Hence, eight feature vectors are chosen using these networks and the top 256 features of these networks are chosen using neighborhood component analysis (NCA). Support vector machine (SVM) classifier has been used for classification. Moreover, by using the eight feature vectors generated, two ensemble models have been presented. In the first ensemble model, generated all features are concatenated and the top 1000 features are chosen using a feature selector used (NCA), and these features are classified using SVM. In the second ensemble model, iterative hard majority voting (IHMV) has been applied to the generated results. The developed ensemble ResNet models attained 98.91% and 99.15% classification accuracies using an SVM classifier with a 10-fold cross-validation strategy. Our results obtained demonstrate the high classification accuracy of our presented ensemble pretrained ResNet-based deep feature extraction models. These developed models are ready to be tested with higher databases before actual real-world application.

MFAN: Multi-Level Features Attention Network for Fake Certificate Image Detection.

Up to now, most of the forensics methods have attached more attention to natural content images. To expand the application of image forensics technology, forgery detection for certificate images that can directly represent people’s rights and interests is investigated in this paper. Variable tampered region scales and diverse manipulation types are two typical characteristics in fake certificate images. To tackle this task, a novel method called Multi-level Feature Attention Network (MFAN) is proposed. MFAN is built following the encoder–decoder network structure. In order to extract features with rich scale information in the encoder, on the one hand, we employ Atrous Spatial Pyramid Pooling (ASPP) on the final layer of a pre-trained residual network to capture the contextual information at different scales; on the other hand, low-level features are concatenated to ensure the sensibility to small targets. Furthermore, the resulting multi-level features are recalibrated on channels for irrelevant information suppression and enhancing the tampered regions, guiding the MFAN to adapt to diverse manipulation traces. In the decoder module, the attentive feature maps are convoluted and unsampled to effectively generate the prediction mask. Experimental results indicate that the proposed method outperforms some state-of-the-art forensics methods.

DeCapsGAN: generative adversarial capsule network for image denoising

A convolutional capsule network that characterizes small objects previously has shown remarkable performance in small object classification. We extend the idea of capsules to the image denoising task and combine it with the generative adversarial network to develop a generative adversarial capsule network (DeCapsGAN). Both the generator and discriminator adopt the capsule network architecture. The convolutional capsule network is used to capture richer image features. We introduce deconvolution into the generator and propose a convolutional–deconvolutional capsule block. Skip connections are beneficial to transfer image features to deeper networks. A pretrained residual network (ResNet) is implemented as a feature extractor that captures features from the denoised image and reference image to measure the difference in perceptual information in the feature space. The performance of the proposed model is evaluated on the image with synthetic noise (Gaussian noise and mixed Gaussian with impulse noise) and real noise. Extensive experiments show that our model achieves excellent denoising performance in terms of both visual quality and quantitative metrics.

Automated psoriasis lesion segmentation from unconstrained environment using residual U-Net with transfer learning

Background and objectiveThe automatic segmentation of psoriasis lesions from digital images is a challenging task due to the unconstrained imaging environment and non-uniform background. Existing conventional or machine learning-based image processing methods for automatic psoriasis lesion segmentation have several limitations, such as dependency on manual features, human intervention, less and unreliable performance with an increase in data, manual pre-processing steps for removal of background or other artifacts, etc. MethodsIn this paper, we propose a fully automatic approach based on a deep learning model using the transfer learning paradigm for the segmentation of psoriasis lesions from the digital images of different body regions of the psoriasis patients. The proposed model is based on U-Net architecture whose encoder path utilizes a pre-trained residual network model as a backbone. The proposed model is retrained with a self-prepared psoriasis dataset and corresponding segmentation annotation of the lesion. ResultsThe performance of the proposed method is evaluated using a five-fold cross-validation technique. The proposed method achieves an average Dice Similarity Index of 0.948 and Jaccard Index of 0.901 for the intended task. The transfer learning provides an improvement in the segmentation performance of about 4.4% and 7.6% in Dice Similarity Index and Jaccard Index metric respectively, as compared to the training of the proposed model from scratch. ConclusionsAn extensive comparative analysis with the state-of-the-art segmentation models and existing literature validates the promising performance of the proposed framework. Hence, our proposed method will provide a basis for an objective area assessment of psoriasis lesions.

Defective egg detection based on deep features and Bidirectional Long-Short-Term-Memory

Eggs are one of the most important nutritional sources worldwide. In addition, defects that may occur in the eggshells endanger food safety and cause adverse effects on production companies. Automatic separation of defective eggs from quality eggs is an important issue due to economic and health reasons. On this motivation, a real-time machine vision system based on deep learning has been developed for the detection of cracked, bloody and dirty eggs. In this study, a continuous rotating system was designed to visualize all surfaces of the egg. Thus, the adverse conditions such as dirt, blood, and cracks that may occur on any surface of the egg have been successfully monitored. In the proposed system for the detection of robust eggs, deep features are extracted using a pre-trained residual network model and then the obtained features are fed into the Bidirectional Long-Short-Term-Memory (BiLSTM). The efficiency of the proposed model was calculated using dirty, bloody, cracked and robust egg images with the developed machine vision system. The experimental works showed that the proposed model achieved a 99.17% accuracy score. The obtained result was also compared with state-of-the-art methods, and the proposed model was observed to exhibit the highest accuracy among the compared methods.

Deep rhythm and long short term memory-based drowsiness detection

In this paper, a deep-rhythm-based approach is proposed for the efficient detection of drowsiness based on EEG recordings. In the proposed approach, EEG images are used instead of signals where the time and frequency information of the EEG signals are incorporated. The EEG signals are converted to EEG images using the time-frequency transformation method. The Short-Time-Fourier-Transform (STFT) is used for this transformation due to its simplicity. The rhythm images are then extracted by dividing the EEG images based on frequency intervals. EEG signals contain five rhythms, namely Delta rhythm (0–4 Hz), Theta rhythm (4–8 Hz), Alpha rhythm (8–12 Hz), Beta rhythm (12–30 Hz), and Gamma rhythm (30–50 Hz). From each rhythm image, deep features are extracted based on a pre-trained convolutional neural network (CNN) model, with pre-trained residual network (ResNet) models such as ResNet18, ResNet50, and ResNet101. The obtained deep features from each rhythm image are fed into the Long-Short-Term-Memory (LSTM) layer, and the LSTM layers are then sequentially connected to each other. After the last LSTM layer, a fully-connected layer, a softmax layer, and a classification layer are employed in order to detect the class labels of the input EEG signals. Various experiments were conducted with the MIT/BIH Polysomnographic Dataset. The experiments showed that the concatenated ResNet features achieved an accuracy score of 97.92%. The obtained accuracy score was also compared with the state-of-the-art scores and, to the best of our knowledge, the proposed method achieved the best accuracy score among the methods compared.

Indoor human activity recognition using high-dimensional sensors and deep neural networks

Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach toward automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pretrained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in combining both sensors to enable activity recognition in the case of less than ideal circumstances.

Visual Emotion Recognition Using ResNet

Given an image, humans have emotional reactions to it such as happy, fear, disgust, etc. The purpose of this research is to classify images based on human's reaction to them using ResNet deep architecture. The problem is that emotional reaction from humans are subjective, therefore a confidently labelled dataset is difficult to obtain. This research tries to overcome this problem by implementing and analyzing transfer learning from a big dataset such as ImageNet to relatively small visual emotion dataset. Other than that, because emotion is determined by low-level and high-level features, we will make a modification to a pretrained residual network to better utilize low-level and high-level feature to be used in visual emotion recognition. Results show that general (low-level) features and specific (high-level) features obtained from ImageNet object recognition can be well utilized for visual emotion recognition.

Residential roof condition assessment system using deep learning

The emergence of high resolution (HR) and ultra high resolution (UHR) airborne remote sensing imagery is enabling humans to move beyond traditional land cover analysis applications to the detailed characterization of surface objects. A residential roof condition assessment method using techniques from deep learning is presented. The proposed method operates on individual roofs and divides the task into two stages: (1) roof segmentation, followed by (2) condition classification of the segmented roof regions. As the first step in this process, a self-tuning method is proposed to segment the images into small homogeneous areas. The segmentation is initialized with simple linear iterative clustering followed by deep learned feature extraction and region merging, with the optimal result selected by an unsupervised index, Q. After the segmentation, a pretrained residual network is fine-tuned on the augmented roof segments using a proposed k-pixel extension technique for classification. The effectiveness of the proposed algorithm was demonstrated on both HR and UHR imagery collected by EagleView over different study sites. The proposed algorithm has yielded promising results and has outperformed traditional machine learning methods using hand-crafted features.