Capsule Network Research Articles

Melanoma Skin Cancer Detection Using Ensemble of Machine Learning Models Considering Deep Feature Embeddings

In today’s AI-driven era, deep learning (DL) algorithms play a crucial role in automatically detecting life-threatening skin cancers, thereby significantly enhancing survival rates. It makes skin cancer detection using DL algorithms an exciting area of exploration. While much of the prior research has focused on single-model approaches, combining ensembles of multiple models can enhance classification accuracy. Previous studies mainly relied on deep convolutional neural networks (DCNNs), which have limitations in capturing global features. Recent advancements have introduced capsule networks (Caps-Net) and vision transformers (ViT) for more effective feature extraction. In our study, we harness the power of DCNN, Caps-Net, and ViT frameworks to extract diverse image embeddings. These obtained feature vectors work as input data to train an ensemble model based on a majority voting mechanism. This ensemble model consists of five machine-learning models, including Random Forest, XGBoost, SVM, KNN, and logistic regression. The incorporation of this ensemble mechanism leads to a significant improvement in the overall model’s performance. It is noteworthy that the proposed ensemble model serves as a lightweight model, which achieves an impressive accuracy of 91.6% when considering the melanoma skin cancer dataset. These results underscore the superiority of our proposed mechanism over individual state-of-the-art (SOTA) models.

Hierarchical Graph Capsule Networks for Molecular Function Classification With Disentangled Representations.

In biochemistry, graph structures have been widely used for modeling compounds, proteins, functional interactions, etc. A common task that divides these graphs into different categories, known as graph classification, highly relies on the quality of the representations of graphs. With the advance in graph neural networks, message-passing-based methods are adopted to iteratively aggregate neighborhood information for better graph representations. These methods, though powerful, still suffer from some shortcomings. The first challenge is that pooling-based methods in graph neural networks may sometimes ignore the part-whole hierarchies naturally existing in graph structures. These part-whole relationships are usually valuable for many molecular function prediction tasks. The second challenge is that most existing methods do not take the heterogeneity embedded in graph representations into consideration. Disentangling the heterogeneity will increase the performance and interpretability of models. This paper proposes a graph capsule network for graph classification tasks with disentangled feature representations learned automatically by well-designed algorithms. This method is capable of, on the one hand, decomposing heterogeneous representations to more fine-grained elements, whilst on the other hand, capturing part-whole relationships using capsules. Extensive experiments performed on several public-available biochemistry datasets demonstrated the effectiveness of the proposed method, compared with nine state-of-the-art graph learning methods.

A Plant Leaf Disease Image Classification Method Integrating Capsule Network and Residual Network

A Plant Leaf Disease Image Classification Method Integrating Capsule Network and Residual Network

Person Identification System Using Periocular Biometrics Based on Hybrid Optimal Dense Capsule Network

Person identification using periocular images has emerged as a challenging scenario in efficient biometric analysis, particularly under less constrained environments. Accurate recognition is significant in rendering effective measures during the COVID-19 pandemic. In this research paper, the person identification process is performed based on a deep learning model. Several effectual methods have already been developed, but certain drawbacks still exist, like deteriorated image quality, high computational cost, increased error, less training ability, a requirement of high storage space and accuracy rate degradation. Hence, the proposed work introduces a Hybrid Optimal Dense Capsule network-based Periocular biometric system (HodCP) to conquer these demerits. The proposed work involves pre-processing, dimensionality reduction, hybrid feature extraction and Image matching. The pre-processing step is undertaken using Parabolic Contrast Enhancement (PCE) to balance the image contrast and enhance the image quality. Then the Two-Dimensional Principal Component Analysis (2D_PCA) is employed to minimize the image dimensionality. Deep features are extracted in the hybrid feature extraction process using Dense Convolutional-121 Capsule Network (DenseCapsNet). The net loss and hyperparameter tuning are performed through African Vultures Optimization (AVO) algorithm. Finally, image matching is performed using Weighted Distance Similarity (WDS), which identifies the similarity between the query image and a set of image samples based on the distance score. The simulation tool used for analyzing better performance is PYTHON. The data required to process the proposed work are collected from four benchmark datasets. The proposed work provides a better accuracy rate of CASIA-Iris-Mobile-V1.0 (99.01%), UBIPr (99.12%), Facemask detection dataset (98.67%) and Glasses versus without glasses dataset (98.83%), which is superior to the existing methods.

Image Steganography Using Optimized Twin Attention-Based Convolutional Capsule Network

The secret information hiding inside the cover image is termed image steganography, in which the secret information may be either in visual or text format. The concealment of secret information inside the cover image is devised by converting the information into the standard form using conventional image steganography. Here, the cover image is usually systematically altered to carry the secret binary bits after the translation of secret data into binary bits. The cover image may get distorted due to overload, making the hidden information obvious. As a result, the conventional image steganography approaches have a limited ability to conceal. Hence, this research introduces a novel image steganography using the optimized deep learning technique. For novel image steganography, an improved archerfish hunting optimization-based twin attention convolution capsule network (ImAho-TACCNet) is introduced for image steganography. Here, the proposed ImAho is utilized for modifying the tunable parameters of the TACCNet to enhance the efficiency of the image steganography process in terms of minimum mean square error (MSE) and maximal peak signal-to-noise ratio (PSNR). Besides, secret information compression and recursive encryption techniques further enhance the security of secret information. The analysis of ImAho-TACCNet based on various assessment measures like PSNR, SSIM and MSE accomplished enhanced outcomes with the values of 62.37, 0.9923, and 0.0165 for the hidden network model and 64.23, 0.9989, and 0.0125 for the extraction network model.

Distracted driver behavior recognition using modified capsule networks

Human activity recognition (HAR) is an increasingly active study field within the computer vision community. In HAR, driver behavior can be detected to ensure safe travel. Detect driver behaviors using a capsule network with leave-one-subject-out validation. The study was done using CapsNet with leave-one-subject-out validation to identify driving habits. The proposed method in this study consists of two parts, namely encoder and decoder. The encoder used in this study modifies Sabour’s capsule network architecture by adding a convolution layer before going to the primary capsule layer. The proposed method is evaluated using a primary dataset with 10 classes and 300 images for each class. The dataset is split based on hold-out validation and leave-one-subject-out validation. The resulting models were then compared to conventional CNN architecture. The objective of the research is to identify driving behavior. In this study, the proposed method results an accuracy rate of 97.83 % in the split dataset using hold-out validation. However, the accuracy decreased by 53.11 % when the proposed method was used on a split dataset using leave-one-subject-out validation. This is because the proposed method extracts all features including the attributes of each participant contained in the input image (user-independent). Thus, the resulting model in this study tends to overfit.

Audio-Video Analysis Method of Public Speaking Videos to Detect Deepfake Threat

Aim: The purpose of the article is to present the hypothesis that the use of discrepancies in audiovisual materials can significantly increase the effectiveness of detecting various types of deepfake and related threats. In order to verify this hypothesis, the authors proposed a new method that reveals inconsistencies in both multiple modalities simultaneously and within individual modalities separately, enabling them to effectively distinguish between authentic and altered public speaking videos. Project and methods: The proposed approach is to integrate audio and visual signals in a so-called fine-grained manner, and then carry out binary classification processes based on calculated adjustments to the classification results of each modality. The method has been tested using various network architectures, in particular Capsule networks – for deep anomaly detection and Swin Transformer – for image classification. Pre-processing included frame extraction and face detection using the MTCNN algorithm, as well as conversion of audio to mel spectrograms to better reflect human auditory perception. The proposed technique was tested on multimodal deepfake datasets, namely FakeAVCeleb and TMC, along with a custom dataset containing 4,700 recordings. The method has shown high performance in identifying deepfake threats in various test scenarios. Results: The method proposed by the authors achieved better AUC and accuracy compared to other reference methods, confirming its effectiveness in the analysis of multimodal artefacts. The test results confirm that it is effective in detecting modified videos in a variety of test scenarios which can be considered an advance over existing deepfake detection techniques. The results highlight the adaptability of the method in various architectures of feature extraction networks. Conclusions: The presented method of audiovisual deepfake detection uses fine inconsistencies of multimodal features to distinguish whether the material is authentic or synthetic. It is distinguished by its ability to point out inconsistencies in different types of deepfakes and, within each individual modality, can effectively distinguish authentic content from manipulated counterparts. The adaptability has been confirmed by the successful application of the method in various feature extraction network architectures. Moreover, its effectiveness has been proven in rigorous tests on two different audiovisual deepfake datasets. Keywords: analysis of audio-video stream, detection of deepfake threats, analysis of public speeches

Multiscale capsule networks with attention mechanisms based on domain-invariant properties for cross-domain lifetime prediction

Remaining useful life prediction (RUL) under variable operating conditions has been a hot research topic in engineering. Still, current unsupervised domain adaptive (UDA) methods all use a single metric (MK-MMD or adversarial mechanism), which somewhat improves the cross-domain RUL prediction performance. However, using MK-MMD or the adversarial mechanism alone to measure the inter-domain differences has a single perspective. At the same time, MK-MMD focuses on measuring the differences in the feature distribution layer and fails to capture the differences in the nonlinear relationship between the source and target domains. In addition, learning high-quality degraded features is also a vital issue in RUL prediction. Based on the above points, this paper proposes an attention mechanism multiscale capsule network cross-domain lifetime prediction method (DI-MCNAM) based on domain invariant properties. First, the attention mechanism multiscale capsule network (MCNAM) extracts the deep degradation features of the weighted data. Second, two modules, multilevel domain adaptation and domain classifier, are integrated to reduce inter-domain differences from different perspectives. The maximum information coefficient (MIC) is introduced in the multilevel domain adaptation module. MIC can learn the nonlinear relationship between source and target domain data over time separately, combined with MK-MMD, to reduce the inter-domain feature differences from different levels. Domain classifiers discriminate the confusing domains and reduce inter-domain differences by minimizing the classification power. The performance of the proposed model is validated using publicly available datasets and compared with existing popular methods, and the final results show that the proposed method has high prediction accuracy.

White shark optimizer with optimal deep learning based effective unmanned aerial vehicles communication and scene classification

Unmanned aerial vehicles (UAVs) become a promising enabler for the next generation of wireless networks with the tremendous growth in electronics and communications. The application of UAV communications comprises messages relying on coverage extension for transmission networks after disasters, Internet of Things (IoT) devices, and dispatching distress messages from the device positioned within the coverage hole to the emergency centre. But there are some problems in enhancing UAV clustering and scene classification using deep learning approaches for enhancing performance. This article presents a new White Shark Optimizer with Optimal Deep Learning based Effective Unmanned Aerial Vehicles Communication and Scene Classification (WSOODL-UAVCSC) technique. UAV clustering and scene categorization present many deep learning challenges in disaster management: scene understanding complexity, data variability and abundance, visual data feature extraction, nonlinear and high-dimensional data, adaptability and generalization, real-time decision making, UAV clustering optimization, sparse and incomplete data. the need to handle complex, high-dimensional data, adapt to changing environments, and make quick, correct decisions in critical situations drives deep learning in UAV clustering and scene categorization. The purpose of the WSOODL-UAVCSC technique is to cluster the UAVs for effective communication and scene classification. The WSO algorithm is utilized for the optimization of the UAV clustering process and enables to accomplish effective communication and interaction in the network. With dynamic adjustment of the clustering, the WSO algorithm improves the performance and robustness of the UAV system. For the scene classification process, the WSOODL-UAVCSC technique involves capsule network (CapsNet) feature extraction, marine predators algorithm (MPA) based hyperparameter tuning, and echo state network (ESN) classification. A wide-ranging simulation analysis was conducted to validate the enriched performance of the WSOODL-UAVCSC approach. Extensive result analysis pointed out the enhanced performance of the WSOODL-UAVCSC method over other existing techniques. The WSOODL-UAVCSC method achieved an accuracy of 99.12%, precision of 97.45%, recall of 98.90%, and F1-score of 98.10% when compared to other existing techniques.

A semantic fidelity interpretable-assisted decision model for lung nodule classification.

Early diagnosis of lung nodules is important for the treatment of lung cancer patients, existing capsule network-based assisted diagnostic models for lung nodule classification have shown promising prospects in terms of interpretability. However, these models lack the ability to draw features robustly at shallow networks, which in turn limits the performance of the models. Therefore, we propose a semantic fidelity capsule encoding and interpretable (SFCEI)-assisted decision model for lung nodule multi-class classification. First, we propose multilevel receptive field feature encoding block to capture multi-scale features of lung nodules of different sizes. Second, we embed multilevel receptive field feature encoding blocks in the residual code-and-decode attention layer to extract fine-grained context features. Integrating multi-scale features and contextual features to form semantic fidelity lung nodule attribute capsule representations, which consequently enhances the performance of the model. We implemented comprehensive experiments on the dataset (LIDC-IDRI) to validate the superiority of the model. The stratified fivefold cross-validation results show that the accuracy (94.17%) of our method exceeds existing advanced approaches in the multi-class classification of malignancy scores for lung nodules. The experiments confirm that the methodology proposed can effectively capture the multi-scale features and contextual features of lung nodules. It enhances the capability of shallow structure drawing features in capsule networks, which in turn improves the classification performance of malignancy scores. The interpretable model can support the physicians' confidence in clinical decision-making.

Hybrid D-OCapNet: Automated Multi-Class Alzheimer’s Disease Classification in Brain MRI Using Hybrid Dense Optimal Capsule Network

Efficient detection of Alzheimer’s disease (AD) is challenging in medical image processing. Different methodologies are proposed for detecting AD at earlier stages, but certain demerits emerge, like less robust, less convergent, time-consuming, and lead over maximized losses. Hence the proposed research paper develops an efficient and automated deep learning-based AD detection using MRI image data. The initial step in AD classification is data acquisition which focuses on collecting brain MRI images from Alzheimer’s disease Neuroimaging Initiative (ADNI) Extracted Axial dataset and OASIS dataset. The gathered data are pre-processed through an Improved Median Filter (IMF), normalization is performed using the [0, 1] rescaling method, and skull stripping is done using Morphological Thresholding (MoT). The pre-processed images are fed into Multiview Fuzzy Clustering (MvFC) algorithm to segment the brain tissues as Gray Matter (GM), Cerebrospinal Fluid (CSF) and White Matter (WM) effectively. The process of Deep Feature Extraction, Multi-class Classification and loss optimization is performed using the Hybrid Dense Optimal Capsule Network (Hybrid D-OCapNet). The loss evaluated in the proposed Hybrid D-OCapNet model is optimized using the Modified Bald Eagle Search (M-BES) optimization algorithm. The simulation outcomes of training, testing and validation in AD classification are analyzed using MATLAB. The overall accuracy in classifying AD is 99.32%, sensitivity is 98.42%, specificity is 98.90%, precision is 98.93%, and F1 score is 98.44 for the ADNI dataset. The accuracy of 98.97%, the sensitivity of 98.31% and the F1 score of 98.39% are obtained for the OASIS dataset.

MA-CapsNet-DA: Speech emotion recognition based on MA-CapsNet using data augmentation

Speech emotion recognition (SER) plays a crucial role in Human–computer interaction (HCI) applications. However, it has two challenges: the lack of effectiveness of deep learning models and data scarcity issues. As a result, the deep learning models used in SER would suffer from overfitting seriously. In this study, a novel SER model called Max-avg-pooling capsule network (MA-CapsNet) is proposed and it is an improved capsule network customized for SER. We also adopt Data augmentation (DA) techniques to tackle the data scarcity issue. The proposed MA-CapsNet model consists of five sequential modules: conv-max-pooling, conv-avg-pooling, convolution, primary capsule, and digital capsule module. Furthermore, a new evaluation metric called the Expected accuracy index (EAI) is presented to evaluate the model performance effectively. The proposed approach demonstrates strong advantages over its peer models under different data partition methods, especially for augmented datasets. Experimental results also show that the proposed model has good interpretability in comparison to the peer methods for its less complicated learning topology, relatively smaller parameter sets, and fewer input features.

Weakly labeled sound event detection with a capsule-transformer model

Sound event detection (SED) is a widely studied field that has achieved considerable success. The dynamic routing mechanism of capsule networks has been used for SED, but its performance in capturing global information of audio is still limited. In this paper, we propose a method for SED that by combining the capsule network with transformer leverages the strength of transformer in capturing global features with that of capsule network in capturing local features. The proposed method was evaluated on the DCASE 2017 Task 4 weakly labeled dataset. The obtained F-score and Equal Error Rate are 60.6% and 0.75, respectively. Compared to other baseline systems, our method achieves significantly improved performance. Keywords: Sound event detection, audio tagging, gated convolution, transformer, capsule network.

Accurate multi-interest modeling for sequential recommendation with attention and distillation capsule network

The objective of sequential recommendation systems is to assist users in efficiently discovering relevant information that they need within vast online environments. Existing studies have frequently overlooked users’ diverse interests, failing to capture their multiple latent interests effectively, particularly in cases wherein users have extensive interaction histories. We introduce a novel approach, called Accurate Multi-Interest Modeling for sequential recommendation to address this limitation. The proposed model combines users’ long- and short-term preferences by using an attention-based module, allowing for comprehensive preference modeling. Furthermore, it incorporates a distillation capsule network-based module that models different user interests, with each capsule representing a unique interest. These interests are further refined using an interest distiller to identify the top-k interests of each user. Moreover, a module based on the multi-head self-attention mechanism captures hidden preference transition patterns within user interaction sequences. Finally, a complete user interest representation is obtained by aggregating the outputs of the aforementioned modules. This decoupling-then-fusion strategy allows the model to capture various detailed features better and consider the interactions among different features to model user needs more comprehensively. Experimental results conducted on three datasets unequivocally demonstrate the superiority of our proposed model.

An intelligent deep learning based capsule network model for human detection in indoor surveillance videos

An intelligent deep learning based capsule network model for human detection in indoor surveillance videos

Classification of crop disease-pest questions based on BERT-BiGRU-CapsNet with attention pooling.

Crop disease-pest question classification is an essential part of pest knowledge intelligent question answering system. A crop disease-pest question classification method is proposed on the basis of bidirectional encoder representations from transformers (BERT), bidirectional gated unit (BiGRU), capsule network (CapsNet), and BERT-BiGRU-CapsNet with attention pooling (BBGCAP). In BBGCAP, the unstructured text data are preprocessed vectorically using BERT, BiGRU is used to extract the deep features of the text, attention pooling is used to assign the corresponding weights to the extracted deep information, and CapsNet is used to route the right alternative. BBGCAP is a synthetic model by integrating the advantages of BERT, BiGRU, CapsNet, and attention pooling. The experimental results on the cucumber-pest question database show that the proposed method is superior to the methods based on traditional template matching, support vector machines (SVM), and convolutional neural network-long short-term memory (LSTM), and the accuracy rates of precision, recall, and F1 are all above 902.15%. This method provides technical support for intelligent question answering system of crop disease-pests.

An interactive multi-head self-attention capsule network model for aspect sentiment classification

An interactive multi-head self-attention capsule network model for aspect sentiment classification

Finger vein identification system using capsule networks with hyperparameter tuning

<p>Safety and security systems are essential for personnel who need to be protected and valuables. The security and safety system can be supported using a biometric system to identify and verify permitted users or owners. Finger vein is one type of biometric system that has high-level security. The finger vein biometrics system has two primary functions: identification and verification. Safety and security technology development is often followed by hackers' development of science and technology. Therefore, the science and technology of safety and security need to be continuously developed. The paper proposes finger vein identification using capsule networks with hyperparameter tuning. The augmentation, convolution layer parameters, and capsule layers are optimized. The experimental results show that the capsule network with hyperparameter tuning successfully identifies the finger vein images. The system achieves an accuracy of 91.25% using the Shandong University machine learning and applications-homologous multimodal traits (SDUMLA-HMT) dataset.</p>

Equilibrium Optimization Algorithm with Deep Learning Enabled Prostate Cancer Detection on MRI Images.

The enlargement of the prostate gland in the reproductive system of males is considered a form of prostate cancer (PrC). The survival rate is considerably improved with earlier diagnosis of cancer; thus, timely intervention should be administered. In this study, a new automatic approach combining several deep learning (DL) techniques was introduced to detect PrC from MRI and ultrasound (US) images. Furthermore, the presented method describes why a certain decision was made given the input MRI or US images. Many pretrained custom-developed layers were added to the pretrained model and employed in the dataset. The study presents an Equilibrium Optimization Algorithm with Deep Learning-based Prostate Cancer Detection and Classification (EOADL-PCDC) technique on MRIs. The main goal of the EOADL-PCDC method lies in the detection and classification of PrC. To achieve this, the EOADL-PCDC technique applies image preprocessing to improve the image quality. In addition, the EOADL-PCDC technique follows the CapsNet (capsule network) model for the feature extraction model. The EOA is based on hyperparameter tuning used to increase the efficiency of CapsNet. The EOADL-PCDC algorithm makes use of the stacked bidirectional long short-term memory (SBiLSTM) model for prostate cancer classification. A comprehensive set of simulations of the EOADL-PCDC algorithm was tested on the benchmark MRI dataset. The experimental outcome revealed the superior performance of the EOADL-PCDC approach over existing methods in terms of different metrics.

GCRec: Graph-Augmented Capsule Network for Next-Item Recommendation.

Next-item recommendation has been a hot research, which aims at predicting the next action by modeling users' behavior sequences. While previous efforts toward this task have been made in capturing complex item transition patterns, we argue that they still suffer from three limitations: 1) they have difficulty in explicitly capturing the impact of inherent order of item transition patterns; 2) only a simple and crude embedding is insufficient to yield satisfactory long-term users' representations from limited training sequences; and 3) they are incapable of dynamically integrating long-term and short-term user interest modeling. In this work, we propose a novel solution named graph-augmented capsule network (GCRec), which exploits sequential user behaviors in a more fine-grained manner. Specifically, we employ a linear graph convolution module to learn informative long-term representations of users. Furthermore, we devise a user-specific capsule module and a position-aware gating module, which are sensitive to the relative sequential order of the recently interacted items, to capture sequential patterns at union-level and point-level. To aggregate the long-term and short-term user interests as a representative vector, we design a dual-gating mechanism, which could decide the contribution ratio of each module given different contextual information. Through extensive experiments on four benchmarks, we validate the rationality and effectiveness of GCRec on the next-item recommendation task.