Capsule Neural Network Research Articles

Intelligent diagnostic model for malaria parasite detection and classification using imperative inception-based capsule neural networks

Malaria is an acute fever sickness caused by the Plasmodium parasite and spread by infected Anopheles female mosquitoes. It causes catastrophic illness if left untreated for an extended period, and delaying exact treatment might result in the development of further complications. The most prevalent method now available for detecting malaria is the microscope. Under a microscope, blood smears are typically examined for malaria diagnosis. Despite its advantages, this method is time-consuming, subjective, and requires highly skilled personnel. Therefore, an automated malaria diagnosis system is imperative for ensuring accurate and efficient treatment. This research develops an innovative approach utilizing an urgent, inception-based capsule network to distinguish parasitized and uninfected cells from microscopic images. This diagnostic model incorporates neural networks based on Inception and Imperative Capsule networks. The inception block extracts rich characteristics from images of malaria cells using a pre-trained model, such as Inception V3, which facilitates efficient representation learning. Subsequently, the dynamic imperative capsule neural network detects malaria parasites in microscopic images by classifying them into parasitized and healthy cells, enabling the detection of malaria parasites. The experiment results demonstrate a significant improvement in malaria parasite recognition. Compared to traditional manual microscopy, the proposed system is more accurate and faster. Finally, this study demonstrates the need to provide robust and efficient diagnostic solutions by leveraging state-of-the-art technologies to combat malaria.

Spatiotemporal Capsule Neural Network for Vehicle Trajectory Prediction

Through advancement of the Vehicle-to-Everything (V2X) network, road safety, energy consumption, and traffic efficiency can be significantly improved. An accurate vehicle trajectory prediction benefits communication traffic management and network resource allocation for the real-time application of the V2X network. Recurrent neural networks and their variants have been reported in recent research to predict vehicle mobility. However, the spatial attribute of vehicle movement behavior has been overlooked, resulting in incomplete information utilization. To bridge this gap, we put forward for the first time a hierarchical trajectory prediction structure using the capsule neural network (CapsNet) with three sequential components. First, the geographic information is transformed into a grid map presentation, describing vehicle mobility distribution spatially and temporally. Second, CapsNet serves as the core model to embed local temporal and global spatial correlation through hierarchical capsules. Finally, extensive experiments conducted on actual taxi mobility data collected in Porto city (Portugal) and Singapore show that the proposed method outperforms the state-of-the-art methods.

X-rays imaging analysis for early diagnosis of thoracic disorders using capsule neural network: a deep learning approach

X-rays imaging analysis for early diagnosis of thoracic disorders using capsule neural network: a deep learning approach

MSRCN: A cross-machine diagnosis method for the CNC spindle motors with compound faults

The cross-machine diagnosis of CNC spindle motors with compound faults is essential and challenging because of the subsystem coupling and individual difference. This paper proposed an in-situ fault diagnosis method for cross machine-level individual diagnosis. Plug-and-play modules are specifically designed inspired by signal processing theory, and are embedded into mainstream CNN-based models as an effective industrial diagnostic model, the multiscale spatial–temporal residual capsule neural networks (MSRCN). The internal mechanism of these new modules is explored through ablation experiments and visualization on real industrial motor signals, which shows MSRCN-based models can enrich the multi-scale feature extraction capabilities and benefits the interference resistance of individual related features. In addition, new evaluation operators for degree of confidence are proposed to comprehensively evaluate the performance of deep learning in classification tasks and the reliability of the decision-making.

Detection of the pipeline elbow erosion by percussion and deep learning

Elbows are commonly used in pipelines to change the direction of flow, and the pipeline elbows are prone to erosion caused by the transported medium. Detection of the pipeline elbow erosion is critical to the health of the pipeline system. Currently, most of the detection methods of the pipeline elbow erosion level require the installation of the constant-contact sensor, which are constrained under certain environments. This paper proposes a novel detection method for the pipeline elbow erosion, which is easy-to-implement, low-cost, and free of the installation of a constant-contact-sensor. The proposed method combines percussion, variational mode decomposition (VMD), and deep learning. The percussion-induced audio signal (single-hit) is decomposed into seven modes using VMD, and the multiple random convolutional kernel transform (Multi-Rocket), a machine learning method, is used to select the most representative mode for the single-hit audio signal. Finally, the capsule neural networks, a deep learning method, is utilized to classify the most representative mode under six levels of erosion severity (six classes). To verify the effectiveness of the proposed method, two case studies are conducted under three pipeline elbows with similar structure and dimension. In case study I, all the methods achieve similar classification accuracy (around 100%) under six erosion levels. In case study II, the proposed method (>90%) is far more effective than other shallow learning and deep learning methods (all < 80%) in terms of the classification accuracy. The results demonstrate the superiority of the proposed method in the detection of pipeline elbow erosion levels. To the best of our knowledge, this paper is the first attempt to study the detection of pipeline elbow erosion levels by combining the percussion method and deep learning.

Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption

Capsule neural networks replace simple, scalar-valued neurons with vector-valued capsules. They are motivated by the pattern recognition system in the human brain, where complex objects are decomposed into a hierarchy of simpler object parts. Such a hierarchy is referred to as a parse-tree. Conceptually, capsule neural networks have been defined to mimic this behavior. The capsule neural network (CapsNet), by Sabour, Frosst, and Hinton, is the first actual implementation of the conceptual idea of capsule neural networks. CapsNets achieved state-of-the-art performance on simple image recognition tasks with fewer parameters and greater robustness to affine transformations than comparable approaches. This sparked extensive follow-up research. However, despite major efforts, no work was able to scale the CapsNet architecture to more reasonable-sized datasets. Here, we provide a reason for this failure and argue that it is most likely not possible to scale CapsNets beyond toy examples. In particular, we show that the concept of a parse-tree, the main idea behind capsule neuronal networks, is not present in CapsNets. We also show theoretically and experimentally that CapsNets suffer from a vanishing gradient problem that results in the starvation of many capsules during training.

Tiny Deep Learning Architectures Enabling Sensor-Near Acoustic Data Processing and Defect Localization

The timely diagnosis of defects at their incipient stage of formation is crucial to extending the life-cycle of technical appliances. This is the case of mechanical-related stress, either due to long aging degradation processes (e.g., corrosion) or in-operation forces (e.g., impact events), which might provoke detrimental damage, such as cracks, disbonding or delaminations, most commonly followed by the release of acoustic energy. The localization of these sources can be successfully fulfilled via adoption of acoustic emission (AE)-based inspection techniques through the computation of the time of arrival (ToA), namely the time at which the induced mechanical wave released at the occurrence of the acoustic event arrives to the acquisition unit. However, the accurate estimation of the ToA may be hampered by poor signal-to-noise ratios (SNRs). In these conditions, standard statistical methods typically fail. In this work, two alternative deep learning methods are proposed for ToA retrieval in processing AE signals, namely a dilated convolutional neural network (DilCNN) and a capsule neural network for ToA (CapsToA). These methods have the additional benefit of being portable on resource-constrained microprocessors. Their performance has been extensively studied on both synthetic and experimental data, focusing on the problem of ToA identification for the case of a metallic plate. Results show that the two methods can achieve localization errors which are up to 70% more precise than those yielded by conventional strategies, even when the SNR is severely compromised (i.e., down to 2 dB). Moreover, DilCNN and CapsNet have been implemented in a tiny machine learning environment and then deployed on microcontroller units, showing a negligible loss of performance with respect to offline realizations.

Facial keypoints detection using capsule neural networks

Facial keypoints detection using capsule neural networks

Capsule neural tensor networks with multi-aspect information for Few-shot Knowledge Graph Completion

Few-shot Knowledge Graph Completion (FKGC) has recently attracted significant research interest due to its ability to expand few-shot relation coverage in Knowledge Graphs. Prevailing FKGC approaches focus on exploiting the one-hop neighbor information of entities to enhance few-shot relation embedding. However, these methods select one-hop neighbors randomly and neglect the rich multi-aspect information of entities. Although some methods have attempted to leverage Long Short-Term Memory (LSTM) to learn few-shot relation embedding, they are sensitive to the input order. To address these limitations, we propose the Capsule Neural Tensor Networks with Multi-Aspect Information approach (short for InforMix-FKGC). InforMix-FKGC employs a one-hop neighbor selection strategy based on how valuable they are and encodes multi-aspect information of entities, including one-hop neighbors, attributes and literal description. Then, a capsule network is responsible for integrating the support set and deriving few-shot relation embedding. Moreover, a neural tensor network is used to match the query set with the support set. In this way, InforMix-FKGC can learn few-shot relation embedding more precisely so as to enhance the accuracy of FKGC. Extensive experiments on the NELL-One and Wiki-One datasets demonstrate that InforMix-FKGC significantly outperforms ten state-of-the-art methods in terms of Mean Reciprocal Rank and Hits@K.

AN IMPROVED SEGMENTATION METHOD FOR BRAIN CANCER USING CAPSULE NEURAL NETWORKS

Brain cancer is a life-threatening disease that requires accurate and efficient segmentation methods for effective diagnosis and treatment planning. In this study, we propose an improved segmentation method for brain cancer using Capsule Neural Networks (CapsNets). CapsNets are a promising alternative to traditional convolutional neural networks (CNNs) as they capture spatial relationships between features more effectively. However, existing CapsNet-based segmentation methods suffer from limitations such as low segmentation accuracy and high computational complexity. To address these limitations, we introduce an improved CapsNet architecture that incorporates dynamic routing and attention mechanisms. The dynamic routing algorithm enhances the routing process between capsules, allowing for better feature representation and improved segmentation accuracy. Additionally, the attention mechanism focuses the network’s attention on important regions, reducing the computational complexity without sacrificing segmentation quality. We evaluate the proposed method on a publicly available brain cancer dataset and compare its performance against state-of-the-art segmentation approaches. The experimental results demonstrate that our method achieves superior segmentation accuracy and outperforms existing methods in terms of Dice coefficient and Hausdorff distance. Furthermore, our method demonstrates faster convergence and reduced computational complexity compared to previous CapsNet-based approaches. In conclusion, this study presents an improved segmentation method for brain cancer using Capsule Neural Networks. The proposed method addresses the limitations of existing CapsNet-based approaches by incorporating dynamic routing and attention mechanisms. The experimental results validate the effectiveness of our method, showcasing superior segmentation accuracy and reduced computational complexity. The improved segmentation method has the potential to enhance the diagnosis and treatment planning of brain cancer, ultimately contributing to improved patient outcomes.

COVID-19 Diagnosis in Computerized Tomography (CT) and X-ray Scans Using Capsule Neural Network.

This study proposes a deep-learning-based solution (named CapsNetCovid) for COVID-19 diagnosis using a capsule neural network (CapsNet). CapsNets are robust for image rotations and affine transformations, which is advantageous when processing medical imaging datasets. This study presents a performance analysis of CapsNets on standard images and their augmented variants for binary and multi-class classification. CapsNetCovid was trained and evaluated on two COVID-19 datasets of CT images and X-ray images. It was also evaluated on eight augmented datasets. The results show that the proposed model achieved classification accuracy, precision, sensitivity, and F1-score of 99.929%, 99.887%, 100%, and 99.319%, respectively, for the CT images. It also achieved a classification accuracy, precision, sensitivity, and F1-score of 94.721%, 93.864%, 92.947%, and 93.386%, respectively, for the X-ray images. This study presents a comparative analysis between CapsNetCovid, CNN, DenseNet121, and ResNet50 in terms of their ability to correctly identify randomly transformed and rotated CT and X-ray images without the use of data augmentation techniques. The analysis shows that CapsNetCovid outperforms CNN, DenseNet121, and ResNet50 when trained and evaluated on CT and X-ray images without data augmentation. We hope that this research will aid in improving decision making and diagnostic accuracy of medical professionals when diagnosing COVID-19.

KHGCN: Knowledge-Enhanced Recommendation with Hierarchical Graph Capsule Network

Knowledge graphs as external information has become one of the mainstream directions of current recommendation systems. Various knowledge-graph-representation methods have been proposed to promote the development of knowledge graphs in related fields. Knowledge-graph-embedding methods can learn entity information and complex relationships between the entities in knowledge graphs. Furthermore, recently proposed graph neural networks can learn higher-order representations of entities and relationships in knowledge graphs. Therefore, the complete presentation in the knowledge graph enriches the item information and alleviates the cold start of the recommendation process and too-sparse data. However, the knowledge graph's entire entity and relation representation in personalized recommendation tasks will introduce unnecessary noise information for different users. To learn the entity-relationship presentation in the knowledge graph while effectively removing noise information, we innovatively propose a model named knowledge-enhanced hierarchical graph capsule network (KHGCN), which can extract node embeddings in graphs while learning the hierarchical structure of graphs. Our model eliminates noisy entities and relationship representations in the knowledge graph by the entity disentangling for the recommendation and introduces the attentive mechanism to strengthen the knowledge-graph aggregation. Our model learns the presentation of entity relationships by an original graph capsule network. The capsule neural networks represent the structured information between the entities more completely. We validate the proposed model on real-world datasets, and the validation results demonstrate the model's effectiveness.

Interpretation of a deep analysis of speech imagery features extracted by a capsule neural network

Speech imagery has been successfully employed in developing Brain–Computer Interfaces because it is a novel mental strategy that generates brain activity more intuitively than evoked potentials or motor imagery. There are many methods to analyze speech imagery signals, but those based on deep neural networks achieve the best results. However, more research is necessary to understand the properties and features that describe imagined phonemes and words. In this paper, we analyze the statistical properties of speech imagery EEG signals from the KaraOne dataset to design a method that classifies imagined phonemes and words. With this analysis, we propose a Capsule Neural Network that categorizes speech imagery patterns into bilabial, nasal, consonant-vocal, and vowels/iy/ and/uw/. The method is called Capsules for Speech Imagery Analysis (CapsK-SI). The input of CapsK-SI is a set of statistical features of EEG speech imagery signals. The architecture of the Capsule Neural Network is composed of a convolution layer, a primary capsule layer, and a class capsule layer. The average accuracy reached is 90.88%±7 for bilabial, 90.15%±8 for nasal, 94.02%±6 for consonant–vowel, 89.70%±8 for word-phoneme, 94.33%± for/iy/ vowel and, 94.21%±3 for/uw/ vowel detection. Finally, with the activity vectors of the CapsK-SI capsules, we generated brain maps to represent brain activity in the production of bilabial, nasal, and consonant-vocal signals.

Capsule neural network based approach for subject specific and cross-subjects seizure detection from EEG signals

Capsule neural network based approach for subject specific and cross-subjects seizure detection from EEG signals

Adaptive weight-based capsule neural network for bearing fault diagnosis

In industrial systems, the vibration signals of rolling bearings are highly complex due to varying operating conditions and ambient noise. Presented with signals with a lack of features after being disturbed by a complex environment, traditional convolutional neural networks cannot diagnose bearing faults accurately and effectively. This paper proposes a dynamic capsule network (DCCN) based on adaptive shared weights to solve the above problems. To learn the features of the vibration signals, the convolution weights are adaptively adjusted and shared to different convolutional layers through an attention mechanism, which can effectively reduce the computational cost of the network. In addition, a dynamic routing algorithm is used to generate sub-capsules that share weights. These sub-capsules extract and transform the fault features into vector features information storage to reduce feature loss. Finally, the feature-extraction capability of the DCCN is enhanced by concatenating the dynamic convolution using skip connect lines. The effectiveness of the proposed method is verified by noise and variable-load experiments, and the DCCN exhibits a better generalization and diagnosis performance than current advanced deep-learning methods.

Feature-enhanced text-inception model for Chinese long text classification

To solve the problem regarding unbalanced distribution of multi-category Chinese long texts and improve the classification accuracy thereof, a data enhancement method was proposed. Combined with this method, a feature-enhanced text-inception model for Chinese long text classification was proposed. First, the model used a novel text-inception module to extract important shallow features of the text. Meanwhile, the bidirectional gated recurrent unit (Bi-GRU) and the capsule neural network were employed to form a deep feature extraction module to understand the semantic information in the text; K-MaxPooling was then used to reduce the dimension of its shallow and deep features and enhance the overall features. Finally, the Softmax function was used for classification. By comparing the classification effects with a variety of models, the results show that the model can significantly improve the accuracy of long Chinese text classification and has a strong ability to recognize long Chinese text features. The accuracy of the model is 93.97% when applied to an experimental dataset.

Emotional Speaker Verification Using Novel Modified Capsule Neural Network

Capsule Neural Network (CapsNet) models are regarded as efficient substitutes for convolutional neural networks (CNN) due to their powerful hierarchical representation capability. Nevertheless, CNN endure their inability of recording spatial information in spectrograms. The main constraint of CapsNet is related to the compression method which can be implemented in CNN models but cannot be directly employed in CapsNet. As a result, we propose a novel architecture based on dual-channel long short-term memory compressed CapsNet (DC-LSTM–COMP CapsNet) for speaker verification in emotional as well as stressful talking environments. The proposed approach is perceived as a modified Capsule network that attempts to overcome the limitations that exist within the original CapsNet, as well as in CNN while enhancing the verification performance. The proposed architecture is assessed on four distinct databases. The experimental analysis reveals that the average speaker verification performance is improved in comparison with CNN, the original CapsNet, as well as the conventional classifiers. The proposed algorithm notably achieves the best verification accuracy across the four speech databases. For example, using the Emirati dataset, the average percentage equal error rates (EERs) obtained is 10.50%, based on the proposed architecture which outperforms other deep and classical models.

Intelligent Internet of Things in Mammography Screening Using Multicenter Transformation Between Unified Capsules

Mammography screening is one of the important applications for the intelligent Internet of Things (IoT). Due to the efficient and personalized cyber-medicine system, early diagnosis can successfully reduce the breast cancer mortality rate by AI-driven healthcare. However, it is a huge challenge to extend the conventional single-center into the multicenter mammography screening, thus improving the effectiveness and robustness of intelligent IoT-based devices. To address this problem, we utilize multicenter mammograms by the modified capsule neural network and propose a novel framework called multicenter transformation between unified capsules (MLT-UniCaps) in this article. The proposed MLT-UniCaps is composed of Attentional Pose Embedding, Dynamic Source Capsule Traversal, and Adaptive Target Capsule Fusion to realize an intelligent remote assistant diagnosis. Attentional Pose Embedding extracts feature vectors via variations in position, orientation, scale, and lighting as the poses through an adversarial convolutional neural network with an attention-based layer. Based on the pose presentation, Dynamic Source Capsule Traversal deploys a dynamic routing mechanism between neurons to build a source cancer classifier for single-center mammography screening. Using the source cancer classifier, Adaptive Target Capsule Fusion integrates various centers of mammograms as the universal cancer detectors and optimizes heterogeneous distribution among them by the transformation-likelihood maximization. Owing to the three components, MLT-UniCaps effectively improves the results of single-center mammography screening and works in the multicenter breast cancer diagnosis. By comprehensive experiments on 58 965 samples, the proposed MLT-UniCaps obtains 90.1% of overall classification accuracy on single-center trials and 73.8% of overall F1 score on multicenter trials. All the experimental results illustrated that our MLT-UniCaps, an intelligent IoT-based clinical tool, inures the benefit of mammography screening.

Self-Attentive Channel-Connectivity Capsule Network for EEG-Based Driving Fatigue Detection.

Deep neural networks have recently been successfully extended to EEG-based driving fatigue detection. Nevertheless, most existing models fail to reveal the intrinsic inter-channel relations that are known to be beneficial for EEG-based classification. Additionally, these models require substantial data for training, which is often impractical due to the high cost of data collection. To simultaneously address these two issues, we propose a Self-Attentive Channel-Connectivity Capsule Network (SACC-CapsNet) for EEG-based driving fatigue detection in this paper. SACC-CapsNet starts with a temporal-channel attention module to investigate the critical temporal information and important channels for driving fatigue detection, refining the input EEG signals. Subsequently, the refined EEG data are transformed into a channel covariance matrix to capture the inter-channel relations, followed by selective kernel attention to extract the highly discriminative channel-connectivity features. Finally, a capsule neural network is employed to effectively learn the relationships between connectivity features, which is more suitable for limited data. To confirm the effectiveness of SACC-CapsNet, we collected 24-channel EEG data from 31 subjects (mean age=23.13±2.68 years, male/female=18/13) in a simulated fatigue driving environment. Extensive experiments were conducted with the acquired data, and the comparison results show that our proposed model outperforms state-of-the-art methods. Additionally, the channel covariance matrix learned from SACC-CapsNet reveals that the frontal pole is most informative for detecting driving fatigue, followed by the parietal and central regions. Intriguingly, the temporal-channel attention module can enhance the significance of these critical regions, and the reconstructed channel covariance matrix generated by the decoder network of SACC-CapsNet can effectively preserve valuable information about them.

A Hybrid Approach for Plant Disease Detection Using E-GAN and CapsNet

Crop protection is a great obstacle to food safety, with crop diseases being one of the most serious issues. Plant diseases diminish the quality of crop yield. To detect disease spots on grape leaves, deep learning technology might be employed. On the other hand, the precision and efficiency of identification remain issues. The quantity of images of ill leaves taken from plants is often uneven. With an uneven collection and few images, spotting disease is hard. The plant leaves dataset needs to be expanded to detect illness accurately. A novel hybrid technique employing segmentation, augmentation, and a capsule neural network (CapsNet) is used in this paper to tackle these challenges. The proposed method involves three phases. First, a graph-based technique extracts leaf area from a plant image. The second step expands the dataset using an Efficient Generative Adversarial Network E-GAN. Third, a CapsNet identifies the illness and stage. The proposed work has experimented on real-time grape leaf images which are captured using an SD1000 camera and PlantVillage grape leaf datasets. The proposed method achieves an effective classification of accuracy for disease type and disease stages detection compared to other existing models.