Classical Architecture Research Articles

Hybrid Vision Transformer and Convolutional Neural Network for Multi-Class and Multi-Label Classification of Tuberculosis Anomalies on Chest X-Ray

Tuberculosis (TB), caused by Mycobacterium tuberculosis, remains a leading cause of global mortality. While TB detection can be performed through chest X-ray (CXR) analysis, numerous studies have leveraged AI to automate and enhance the diagnostic process. However, existing approaches often focus on partial or incomplete lesion detection, lacking comprehensive multi-class and multi-label solutions for the full range of TB-related anomalies. To address this, we present a hybrid AI model combining vision transformer (ViT) and convolutional neural network (CNN) architectures for efficient multi-class and multi-label classification of 14 TB-related anomalies. Using 133 CXR images from Dr. Cipto Mangunkusumo National Central General Hospital and 214 images from the NIH datasets, we tackled data imbalance with augmentation, class weighting, and focal loss. The model achieved an accuracy of 0.911, a loss of 0.285, and an AUC of 0.510. Given the complexity of handling not only multi-class but also multi-label data with imbalanced and limited samples, the AUC score reflects the challenging nature of the task rather than any shortcoming of the model itself. By classifying the most distinct TB-related labels in a single AI study, this research highlights the potential of AI to enhance both the accuracy and efficiency of detecting TB-related anomalies, offering valuable advancements in combating this global health burden.

Decorrelative network architecture for robust electrocardiogram classification

Decorrelative network architecture for robust electrocardiogram classification

Bone fracture classification using convolutional neural network architecture for high-accuracy image classification

Bone fracture classification using convolutional neural network architecture for high-accuracy image classification

To crop or not to crop: Comparing whole‐image and cropped classification on a large dataset of camera trap images

AbstractCamera traps facilitate non‐invasive wildlife monitoring, but their widespread adoption has created a data processing bottleneck: a camera trap survey can create millions of images, and the labour required to review those images strains the resources of conservation organisations. AI is a promising approach for accelerating image review, but AI tools for camera trap data are imperfect; in particular, classifying small animals remains difficult, and accuracy falls off outside the ecosystems in which a model was trained. It has been proposed that incorporating an object detector into an image analysis pipeline may help address these challenges, but the benefit of object detection has not been systematically evaluated in the literature. In this work, the authors assess the hypothesis that classifying animals cropped from camera trap images using a species‐agnostic detector yields better accuracy than classifying whole images. We find that incorporating an object detection stage into an image classification pipeline yields a macro‐average F1 improvement of around 25% on a large, long‐tailed dataset; this improvement is reproducible on a large public dataset and a smaller public benchmark dataset. The authors describe a classification architecture that performs well for both whole and detector‐cropped images, and demonstrate that this architecture yields state‐of‐the‐art benchmark accuracy.

Supervised multiple kernel learning approaches for multi-omics data integration

BackgroundAdvances in high-throughput technologies have originated an ever-increasing availability of omics datasets. The integration of multiple heterogeneous data sources is currently an issue for biology and bioinformatics. Multiple kernel learning (MKL) has shown to be a flexible and valid approach to consider the diverse nature of multi-omics inputs, despite being an underused tool in genomic data mining.ResultsWe provide novel MKL approaches based on different kernel fusion strategies. To learn from the meta-kernel of input kernels, we adapted unsupervised integration algorithms for supervised tasks with support vector machines. We also tested deep learning architectures for kernel fusion and classification. The results show that MKL-based models can outperform more complex, state-of-the-art, supervised multi-omics integrative approaches.ConclusionMultiple kernel learning offers a natural framework for predictive models in multi-omics data. It proved to provide a fast and reliable solution that can compete with and outperform more complex architectures. Our results offer a direction for bio-data mining research, biomarker discovery and further development of methods for heterogeneous data integration.

Anomaly-based network intrusion detection using denoising autoencoder and Wasserstein GAN synthetic attacks

Intrusion detection systems face challenges in handling high-dimensional, large-scale, and imbalanced network traffic data. This paper proposes a novel architecture combining a denoising autoencoder (AE) and a Wasserstein generative adversarial network (WGAN) to address these issues. The AE-WGAN model extracts high-representative features and generates realistic synthetic attacks, effectively resolving data imbalance and enhancing anomaly-based intrusion detection. Extensive experiments on NSL-KDD and CICIDS-2017 datasets, using both binary and multiclass classification scenarios with various classifier architectures, demonstrate the model’s superior performance. The proposed approach outperforms state-of-the-art models in accuracy, precision, recall, and F1 score, showing excellent generalization capabilities against unseen attacks. Time complexity analysis reveals computational efficiency while maintaining high-quality synthetic attack generation. This research contributes a robust, efficient, and adaptable framework for intrusion detection, capable of handling modern network traffic complexities and evolving cyber threats.

UniBioPAN: A Novel Universal Classification Architecture for Bioactive Peptides Inspired by Video Action Recognition.

The classification of bioactive peptides is of great importance in protein biology, but there is still a lack of a universal and effective classifier. Inspired by video action recognition, we developed the UniBioPAN architecture to create a universal peptide classifier to solve this problem. The architecture treats the peptide sequence as a video sequence and the molecular image of each amino acid in the peptide sequence as a video frame, enabling feature extraction and classification using convolutional neural networks, bidirectional long short-term memory networks, and fully connected networks. As a novel peptide classification architecture, UniBioPAN significantly outperforms other universal architecture in ACC, AUC and MCC across 11 data sets, and F1 score in 9 data sets. UniBioPAN is available in three ways: python script, jupyter notebook script and web server (https://gzliang.cqu.edu.cn/software/UniBioPAN.html). In summary, UniBioPAN is a universal, convenient, and high-performance peptide classification architecture. UniBioPAN holds significant importance in the discovery of bioactive peptides and the advancement of peptide classifiers. All the codes and data sets are publicly available at https://github.com/sanwrh/UniBioPAN.

Analysis of the effectiveness of using U-net architecture for classification and segmentation of glioma in MRI images

The purpose of the research is to analyze the efficiency of the U-net neural network architecture in decision support systems for glioma diagnostics and segmentation of brain areas affected by it on MRI images.Methods. To conduct experimental studies, a training dataset was generated and the data was normalized. A software implementation of the U-Net neural network architecture was performed using the Keras framework in the Python programming language. The neural network model was trained.Results. A series of experiments were conducted, during which error and classification matrices were obtained, the efficiency of classification of the trained neural network model for the "Tumor" and "No tumor" classes was assessed using metrics such as Recall, Precision and F1-measure, and the quality of segmentation of glioma-affected areas on the test data set was assessed. The quality of segmentation was assessed using the IoU metric, which reflects the ratio of the areas of the bounding boxes and is used to assess the accuracy of the spatial correspondence of the predicted segmented areas highlighted on the masks. Based on the results of testing the neural network model in solving the problem of segmenting brain areas affected by glioma, the average value of the IoU metric was 0.812, which is an acceptable result.Conclusion. The testing results showed that the neural network model based on the U-net architecture is able to effectively diagnose the presence of glioma with acceptable values of the classification and segmentation quality metrics, which indicates the possibility of using this neural network model in medical decision support systems for glioma diagnostics, as well as its segmentation on MRI images. However, it is advisable to refine this neural network model to reduce the number of false negative classification results, which is critically important in medical diagnostics.

Analysis of Quantum-Classical Hybrid Deep Learning for 6G Image Processing with Copyright Detection

This study investigates the integration of quantum computing, classical methods, and deep learning techniques for enhanced image processing in dynamic 6G networks, while also addressing essential aspects of copyright technology and detection. Our findings indicate that quantum methods excel in rapid edge detection and feature extraction but encounter difficulties in maintaining image quality compared to classical approaches. In contrast, classical methods preserve higher image fidelity but struggle to satisfy the real-time processing requirements of 6G applications. Deep learning techniques, particularly CNNs, demonstrate potential in complex image analysis tasks but demand substantial computational resources. To promote the ethical use of AI-generated images, we introduce copyright detection mechanisms that employ advanced algorithms to identify potential infringements in generated content. This integration improves adherence to intellectual property rights and legal standards, supporting the responsible implementation of image processing technologies. We suggest that the future of image processing in 6G networks resides in hybrid systems that effectively utilize the strengths of each approach while incorporating robust copyright detection capabilities. These insights contribute to the development of efficient, high-performance image processing systems in next-generation networks, highlighting the promise of integrated quantum-classical–classical deep learning architectures within 6G environments.

Exploration of an intrinsically explainable self-attention based model for prototype generation on single-channel EEG sleep stage classification

Prototype-based methods in deep learning offer interpretable explanations for decisions by comparing inputs to typical representatives in the data. This study explores the adaptation of SESM, a self-attention-based prototype method successful in electrocardiogram (ECG) tasks, for electroencephalogram (EEG) signals. The architecture is evaluated on sleep stage classification, exploring its efficacy in predicting stages with single-channel EEG. The model achieves comparable test accuracy compared to EEGNet, a state-of-the-art black-box architecture for EEG classification. The generated prototypical components are exaimed qualitatively and using the area over the perterbation curve (AOPC) indicate some alignment with expected bio-markers for different sleep stages such as alpha spindles and slow waves in non-REM sleep, but the results are severely limited by the model’s ability to only extract and present information in the time-domain. Ablation studies are used to explore the impact of kernel size, number of heads, and diversity threshold on model performance and explainability. This study represents the first application of a self-attention based prototype method to EEG data and provides a step forward in explainable AI for EEG data analysis.

Ensemble Pretrained Convolutional Neural Networks for the Classification of Insulator Surface Conditions

Overhead transmission line insulators are non-conductive materials that separate conductors from grounded transmission towers. Once in operation, they frequently experience environmental pollution and electrical or mechanical stress. Since adverse operational conditions can lead to insulation failure, regular inspections are essential to prevent power outages. To this end, this paper proposes a novel technique based on deep convolutional neural networks (CNNs) to classify high-voltage insulator surface conditions based on their image. Successful applications of CNNs in computer vision have led to several pretrained architectures for image classification. To use these pretrained models, a practitioner typically fine-tunes and selects one final model via a model selection stage and discards all other models. In contrast with many existing studies that use such a “winner-takes-all” approach, here, we identify the best subset of seven popular pretrained CNN architectures that are combined by soft voting to form an ensemble classifier. From a machine learning (ML) perspective, this focus is warranted because the convolutional base of each pretrained architecture operates as a feature extractor and an ensemble of them works as a combination of various feature extraction rules. Our numerical experiments demonstrate the advantage of the identified ensemble model to individual pretrained architectures.

Point cloud segmentation neural network with same-type point cloud assistance

This paper proposes neural network architectures for point cloud segmentation, which leverage prior knowledge derived from same-type point clouds. The approach involves concurrent processing of two point clouds: a target point cloud necessitating segmentation and a labeled same-type point cloud. The labeled point cloud provides preliminary labeling information, assisting in segmenting the target point cloud. A feature combination module is proposed to identify and combine corresponding features across the point clouds. The module augments the feature representation of the target cloud and improves its capacity for object discrimination. Experiments on the ShapeNetPart and S3DIS datasets demonstrate that when integrated into classical network architectures, the proposed approach can achieve improved segmentation performance over the corresponding networks, significantly in some of them.

TST-Refrac: A Novel Transformer Variant for Prediction of Production of Re-fractured Oil Wells

Abstract The accurate prediction of post-fracturing production for re-fractured wells is crucial for selecting suitable targets for re-fracturing. Given that the post-fracturing production of re-fractured wells is essentially time series data, this study aims to provide an accurate deep neural network approach for production prediction of re-fractured wells. Traditional deep neural networks, such as Recurrent Neural Networks (RNN) and Long Short-Term Memory Networks (LSTM), are not adept at capturing long-range dependencies. To address this limitation, a new Transformer Variant for the Prediction of Production of Re-fractured Oil Wells is proposed, based on the time window method and the classic Transformer architecture. Experimental results demonstrate that the performance of the Transformer shows significant improvement compared to RNN and LSTM. Utilizing production data from Block W in the J Basin, a random sample of nine wells was selected for model fitting and prediction. The time series Transformer model exhibited the lowest Root Mean Square Error. The successful implementation of the proposed Transformer model for time series demonstrates its capability to effectively capture long-range dependencies, enabling more precise predictions than traditional deep learning methods.

Speech Emotion Recognition Algorithm of Intelligent Robot Based on ACO-SVM

Despite the significant advancements in computer speech emotion recognition technology, the deployment of intelligent robots in this domain continues to encounter challenges related to inefficiency and emotional ambiguity. Aiming at the shortcomings of existing algorithms in the accuracy and processing of complex emotion states, a novel emotion recognition model of intelligent robot speech based on ACO algorithm and multi-level support vector machine is proposed. This paper first identifies the efficiency and accuracy problems existing in the current emotion recognition system, and clarified the research gap, that is, how to effectively improve the recognition performance of the model under diverse emotion states. To solve this problem, genetic algorithm is used to fine-tune SVM model parameters, and multi-level SVM classification architecture is constructed to enhance the accuracy of emotion recognition. In the feature selection stage, ACO algorithm is introduced to explore the optimal combination of emotion features, aiming at improving the efficiency and robustness of emotion recognition. The experimental results showed that the average recognition rate of the multilevel SVM model optimized by genetic algorithm was as high as 81.26%, which was 8.78 percentage points higher than that before optimization. When the ACO algorithm was combined with the optimized multi-stage SVM, the average recognition rate of German and Chinese speech emotion corpus was further improved to 85.62%, with an improvement of 4.38 percentage points. In particular, the optimized algorithm performed significantly better than other emotion categories in recognizing complex emotions such as "anger" and "sadness." This paper not only offers an innovative solution for the emotion recognition technology within the realm of intelligent robots, but also extends its methodology to have significant implications for related fields, thereby strongly advancing the in-depth application and development of intelligent bionic algorithms in speech emotion recognition. In addition, through this series of optimization strategies, this paper significantly improves the accuracy of emotion recognition, laying a solid foundation for intelligent robots to achieve more natural and efficient human-computer interaction.

Real-time modulation classification architecture in software defined radio

Real-time automatic modulation classification (AMC) in software-defined radio (SDR), an emerging approach to blindly detect the modulation scheme in future wireless networks with applications ranging from civilian to military, has not received much attention with practical receiver architecture. This paper proposes a simple modified receiver architecture using reduced Convolutional Neural Networks (CNN) to achieve AMC, which is tested on SDR devices under the bit-error-rate (BER) metric. The results show that the proposed receiver does not change the receiver’s original architecture in practice while achieving good BER performance. The results also show that the proposed low-complexity CNN model can fasten running time, suitable for real-time AMC applications.

Learning and consolidating the contextualized contour representations of tones from F0 sequences and durational variations via transformers

Many speech characteristics, including conventional acoustic features such as mel frequency cepstrum coefficients and mel-spectrograms, as well as pre-trained contextualized acoustic representations such as wav2vec2.0, are used in a deep neural network or successfully fine-tuned with a connectionist temporal classification for Mandarin tone classification. In this study, the authors propose a transformer-based tone classification architecture, TNet-Full, which uses estimated fundamental frequency (F0) values and aligned boundary information on syllables and words. Key components of the model framework are the contour encoder and rhythm encoder, as well as the cross-attention between contours and rhythms established in the interaction encoder. Using contextual tonal contours as a reference, as well as rhythmic information derived from duration variations to consolidate more on contour representations for tone recognition, TNet-Full achieves absolute performance improvements of 24.4% for read speech (from 71.4% to 95.8%) and 6.3% for conversational speech (from 52.1% to 58.4%) when compared to a naive, simple baseline transformer, TNet-base. The relative improvements are 34.2% and 12.1%. As humans perceive tones, contour abstractions of tones can only be derived from F0 sequences, and tone recognition would be improved if syllable temporal organization was stable and predictable instead of fluctuating as seen in conversations.

Deep Learning Optimization of the EfficienNet Architecture for Classification of Tuberculosis Bacteria

Deep Learning Optimization of the EfficienNet Architecture for Classification of Tuberculosis Bacteria

DeepWhaleNet: Climate Change-Aware FFT-Based Deep Neural Network for Passive Acoustic Monitoring

Climate change poses severe risks to the survival of many whale populations, whose habitats and migration patterns are affected by environmental changes. To detect these whales effectively, especially in deep-sea environments, we need to use AI-based techniques to handle the acoustic diversity and variability of different species. However, current methods for whale detection are based on pre- and post-processing steps that reduce their efficiency and generalizability. To address this issue, we present DeepWhaleNet, a novel deep-learning model that automates whale detection in Underwater Passive Acoustic Monitoring datasets. DeepWhaleNet simplifies the detection process by extracting relevant features from raw log-power spectrograms and helps protect these threatened species by supporting conservation efforts. Our model uses a larger short-time Fourier transform as input and a custom ResNet-18 architecture for classification, which enables it to separate whale sounds from noise and capture their temporal and spectral characteristics. We evaluate the performance of DeepWhaleNet and show that it surpasses state-of-the-art methods, achieving an 8.3% improvement in the F-1 score and 21% higher average precision of binary relevance than the baseline method. Moreover, our model demonstrates its versatility and suitability for species-specific retrieval problems through an ablation study on multi-label retrieval problems and a 99.1% recall for Blue Whales.

Impact of Pre-trained Weights on MobileNet Architectures for Animal Classification

This study aims to address the problem of improving animal image classification accuracy using different versions of MobileNet. Accurate animal classification plays a vital role in biodiversity protection, environmental monitoring, and agriculture. The research is significant because existing studies focus on specific models and datasets, leaving a gap in the comparative performance analysis of MobileNet versions. To address this issue, MobileNet V1, V2, and V3 models were utilized, both with and without ImageNet pre-trained weights. The models were trained on a dataset composed of 30,179 images from two sources, covering 13 animal categories. The experiment involved training the models over 10 epochs using a standard configuration of the TensorFlow framework, with accuracy serving as the primary evaluation metric.The results showed that MobileNet V3Large, with pre-trained weights, achieved the highest accuracy (97.43%), outperforming V1 and V2. Using pre-trained weights consistently enhanced performance, as models without pre-training exhibited lower accuracy and slower convergence. This study contributes by providing a comprehensive comparison of MobileNet versions in animal classification tasks, demonstrating the importance of pre-training and model architecture optimization for achieving high accuracy in image classification.

Rice Leaf Disease Classification—A Comparative Approach Using Convolutional Neural Network (CNN), Cascading Autoencoder with Attention Residual U-Net (CAAR-U-Net), and MobileNet-V2 Architectures

Classifying rice leaf diseases in agricultural technology helps to maintain crop health and to ensure a good yield. In this work, deep learning algorithms were, therefore, employed for the identification and classification of rice leaf diseases from images of crops in the field. The initial algorithmic phase involved image pre-processing of the crop images, using a bilateral filter to improve image quality. The effectiveness of this step was measured by using metrics like the Structural Similarity Index (SSIM) and the Peak Signal-to-Noise Ratio (PSNR). Following this, this work employed advanced neural network architectures for classification, including Cascading Autoencoder with Attention Residual U-Net (CAAR-U-Net), MobileNetV2, and Convolutional Neural Network (CNN). The proposed CNN model stood out, since it demonstrated exceptional performance in identifying rice leaf diseases, with test Accuracy of 98% and high Precision, Recall, and F1 scores. This result highlights that the proposed model is particularly well suited for rice leaf disease classification. The robustness of the proposed model was validated through k-fold cross-validation, confirming its generalizability and minimizing the risk of overfitting. This study not only focused on classifying rice leaf diseases but also has the potential to benefit farmers and the agricultural community greatly. This work highlights the advantages of custom CNN models for efficient and accurate rice leaf disease classification, paving the way for technology-driven advancements in farming practices.