Network Transformation Research Articles

Review of Convolutional Neural Network

Over the past decade, the domain of deep learning has emerged as a swiftly expanding area of interest among researchers globally. It offers substantial benefits over traditional shallow networks, particularly in the realms of feature extraction and model fitting. Deep learning excels at uncovering intricate, distributed features from raw data, which exhibit robust generalization capabilities. It has triumphantly addressed challenges that were once deemed intractable within the field of artificial intelligence. With the exponential growth in the volume of training data and a marked increase in computational power, deep learning has achieved remarkable strides and found applications across a spectrum of domains, including but not limited to object detection, computer vision, natural language processing, speech recognition, and semantic parsing, thereby accelerating the evolution of artificial intelligence. Deep learning encompasses a series of non-linear transformations organized hierarchically, with deep neural networks representing the predominant form of contemporary deep learning methodologies. Inspired by the neural connections found in the animal visual cortex, these networks are characterized by local connectivity, shared weights, and pooling operations. Such attributes not only simplify the network model and curtail the number of trainable parameters but also engender invariance to shifts, distortions, and scaling, thereby endowing the model with enhanced robustness and fault tolerance. This makes the training and optimization of the network structure more manageable. This paper commences with a retrospective on the evolution of convolutional neural networks (CNNs). Subsequently, it delineates the architectures of neuron models and multilayer perceptron. It proceeds to dissect the CNN architecture, which typically encompasses a sequence of convolutional and pooling layers, culminating in fully connected layers, each serving distinct functions. The paper also elaborates on various enhanced algorithms for CNNs, such as the “Network in Network” and “Spatial Transformer Networks,” while also introducing supervised and unsupervised learning methodologies pertinent to CNNs and highlighting several prevalent open-source tools. Further, the paper scrutinizes the application of CNNs in various fields, including image classification, facial recognition, audio retrieval, electrocardiogram analysis, and object detection. It posits the amalgamation of CNNs with recurrent neural networks as a potential alternative for training datasets. The research culminates in the design of CNN structures with varying parameters and depths, supported by experimental analysis that elucidates the interplay among these parameters and the ramifications of their configuration. Ultimately, the paper synthesizes the merits and lingering challenges associated with CNNs and their practical applications.

Diffusion-driven Incomplete Multimodal Learning for Air Quality Prediction

Predicting air quality using multimodal data is crucial to comprehensively capture the diverse factors influencing atmospheric conditions. Therefore, this study introduces a multimodal learning framework that integrates outdoor images with traditional ground-based observations to improve the accuracy and reliability of air quality predictions. However, aligning and fusing these heterogeneous data sources pose a formidable challenge, further exacerbated by pervasive data incompleteness issues in practice. In this paper, we propose a novel incomplete multimodal learning approach (iMMAir) to recovery missing data for robust air quality prediction. Specifically, we first design a shallow feature extractor to capture modal-specific features within the latent representation space. Then we develop a conditional diffusion-driven recovery module to mitigate the distribution gap between the recovered and true data. This module further incorporates two conditional constraints of temporal correlation and semantic consistency for effective modal completion. Finally, we reconstruct incomplete modalities and fuse available data using a multimodal transformer network to predict the air quality. To alleviate the modality imbalance problem, we employ an adaptive gradient modulation strategy to adjust the optimization of each modality. Experimental results demonstrate that iMMAir significantly reduces prediction errors, outperforming baseline models by an average of 5.6% and 2.5% in air quality regression and classification tasks. Our source code and data are available at https://github.com/pestasu/IMMAir.

DELINEATE-Regurgitation: deep learning for automated assessment of aortic, mitral, and tricuspid regurgitation from echocardiography

Abstract Background Valvular heart diseases, including aortic regurgitation (AR), mitral regurgitation (MR), and tricuspid regurgitation (TR) are significant contributors to cardiovascular morbidity and mortality worldwide. Artificial intelligence (AI) may improve echocardiographic assessment, but no models to date have successfully evaluated valvular regurgitation. Purpose The goal of this study was to develop an AI system to improve the accuracy of echocardiographic AR, MR, and TR interpretation. Methods Complete transthoracic echocardiogram (TTE) studies performed at a single center were split into train (n=65,301), validation (14,018), and test sets (13,746). The Deep Learning for Echo Analysis, Tracking, and Evaluation of Valvular Regurgitation (DELINEATE-Regurgitation) system intakes entire TTE studies, identifies color Doppler clips showing each valve, and yields a study level classification of AR, MR, and TR on a 6-grade scale (none/trace, mild, mild-moderate, moderate, moderate-severe, severe) using the cardiologist interpretation as a reference standard. This system used a hybrid neural network, leveraging the spatiotemporal feature learning capabilities of convolutional neural networks with the sequential processing strengths of transformer networks. The end-to-end hybrid approach allows the model to learn nuanced representations of the data including features within and across color Doppler videos. Model accuracy was assessed using quadratically weighted Cohen’s kappa (k), area under the receiver operator characteristic curve (AUROC) for the detection of both "moderate or greater" and "severe" regurgitation, and AI-cardiologist agreement the 6-grade scale. Additionally, an analysis was conducted using a 4-grade regurgitation scale (none/trace, mild, moderate, and severe) by rolling up intermediate classes (mild-moderate and moderate-severe) into the next higher class for both cardiologist determinations and model predictions. Results The DELINEATE-Regurgitation system demonstrated a high accuracy in the classification of AR (k=0.857), MR (k=0.867), and TR (k=0.847) (Figure 1). DELINEATE predicted the same severity as cardiologists (AR 91.8%, MR 79.9%, TR 77.7%) or within ±1 grade accuracy (AR 99.5%, MR 98.6%, TR 98.6%). The models had excellent binary discrimination in the detection of moderate or greater (AR AUROC 99.4, MR 98.6, TR 98.1) and severe (AR 99.7, MR 99.3, TR 99.6) regurgitation (Figure 2). In the 4-grade scale, the model had exact agreement with cardiologists in 92.3% of AR, 82.4% of MR, and 80.6% of TR cases. Panel adjudication of AI-cardiologist disagreement and correlation with quantitative metrics at higher levels of regurgitation will be presented. Conclusions The DELINEATE-Regurgitation AI system demonstrates a high level of accuracy in the assessment of AR, MR, and TR using color Doppler TTE videos. Future work will focus on automating quantification and developing strategies for optimal clinical deployment.Figure 1 - Confusion MatricesFigure 2 - ROC Curves

WiTUnet: A U-shaped architecture integrating CNN and Transformer for improved feature alignment and local information fusion

Low-dose computed tomography (LDCT) has emerged as the preferred technology for diagnostic medical imaging due to the potential health risks associated with X-ray radiation and conventional computed tomography (CT) techniques. While LDCT utilizes a lower radiation dose compared to standard CT, it results in increased image noise, which can impair the accuracy of diagnoses. To mitigate this issue, advanced deep learning-based LDCT denoising algorithms have been developed. These primarily utilize Convolutional Neural Networks (CNNs) or Transformer Networks and often employ the Unet architecture, which enhances image detail by integrating feature maps from the encoder and decoder via skip connections. However, existing methods focus excessively on the optimization of the encoder and decoder structures while overlooking potential enhancements to the Unet architecture itself. This oversight can be problematic due to significant differences in feature map characteristics between the encoder and decoder, where simple fusion strategies may hinder effective image reconstruction. In this paper, we introduce WiTUnet, a novel LDCT image denoising method that utilizes nested, dense skip pathway in place of traditional skip connections to improve feature integration. Additionally, to address the high computational demands of conventional Transformers on large images, WiTUnet incorporates a windowed Transformer structure that processes images in smaller, non-overlapping segments, significantly reducing computational load. Moreover, our approach includes a Local Image Perception Enhancement (LiPe) module within both the encoder and decoder to replace the standard multi-layer perceptron (MLP) in Transformers, thereby improving the capture and representation of local image features. Through extensive experimental comparisons, WiTUnet has demonstrated superior performance over existing methods in critical metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Root Mean Square Error (RMSE), significantly enhancing noise removal and image quality. The code is available on github https://github.com/woldier/WiTUNet.

Reference-based image super-resolution of hyperspectral and red-green-blue image for determination of wheat kernel quality using deep learning networks

In the process of cultivation and harvest, wheat kernel quality is highly susceptible to various factors, such as disease, mildew, atrophy and impurities, and detection of kernel quality is essential to avoid hazard proliferation, facilitate product grading, and ensure food safety. Possessing abundant image and spectral characteristics, hyperspectral imaging (HSI) has gained impressive achievements in kernel quality analysis, but its low spatial resolution limits its detection accuracy. In this study, reference-based image super-resolution (RefSR) of HSI and Red-Green-Blue image was adopted to improve resolution to determine wheat kernel quality using deep learning networks. Firstly, RefSR was conducted by the improved transformer network with dual-branch feature extraction and weighted fusion operation and achieved excellent RefSR with significant resolution improvement, peak signal to noise ratio of 35.521 and structural similarity index of 0.97, outweighing the existing state-of-the-art networks. Then, the reflectance images (RIs) of effective wavelengths (EWs) from generated HSI images were combined with the residual network with a spatial, channel attention and multi-scale residual to determine wheat kernel quality. Precise analysis was achieved with the accuracy in calibration, validation and prediction sets of 100.00%, 95.26% and 92.78%. RefSR provides a novel and efficient approach for obtaining HSI images of high spatial resolution and facilitates the application of HSI in analysis of crop kernels. RIs of several sporadic EWs can be easily acquired and processed, achieving field and rapid kernel detection. Therefore, the proposed method furnishes the efficient, accurate and applicable determination of wheat kernel quality.

GTasm: a genome assembly method using graph transformers and HiFi reads

MotivationGenome assembly aims to reconstruct the whole chromosome-scale genome sequence. Obtaining accurate and complete chromosome-scale genome sequence serve as an indispensable foundation for downstream genomics analyses. Due to the complex repeat regions contained in genome sequence, the assembly results commonly are fragmented. Long reads with high accuracy rate can greatly enhance the integrity of genome assembly results.ResultsHere we introduce GTasm, an assembly method that uses graph transformer network to find optimal assembly results based on assembly graphs. Based on assembly graph, GTasm first extracts features about vertices and edges. Then, GTasm scores the edges by graph transformer model, and adopt a heuristic algorithm to find optimal paths in the assembly graph, each path corresponding to a contig. The graph transformer model is trained using simulated HiFi reads from CHM13, and GTasm is compared with other assembly methods using real HIFI read set. Through experimental result, GTasm can produce well assembly results, and achieve good performance on NA50 and NGA50 evaluation indicators. Applying deep learning models to genome assembly can improve the continuity and accuracy of assembly results. The code is available from https://github.com/chu-xuezhe/GTasm.

Bidirectional-enhanced transformer network with channel weighting feature fusion for remote sensing image change detection

Bidirectional-enhanced transformer network with channel weighting feature fusion for remote sensing image change detection

TRGCN: A Prediction Model for Information Diffusion Based on Transformer and Relational Graph Convolutional Network

In order to capture and integrate the structural features and temporal features contained in social graph and diffusion cascade more effectively, an information diffusion prediction model based on the Transformer and Relational Graph Convolutional Network (TRGCN) is proposed. Firstly, a dynamic heterogeneous graph composed of the social network graph and the diffusion cascade graph was constructed, and it was input into the Relational Graph Convolutional Network (RGCN) to extract the structural features of each node. Secondly, the time embedding of each node was reencoded using Bi-directional Long Short-Term Memory (Bi-LSTM). The time decay function was introduced to give different weights to nodes at different time positions, so as to obtain the temporal features of nodes. Finally, structural features and temporal features were input into Transformer and then merged. The spatiotemporal features are obtained for information diffusion prediction. The experimental results on three real datasets of X (formerly known as Twitter), Douban, and Memetracker show that compared with the optimal model in the comparison experiment, the TRGCN model has an average increase of 4.16% in Hits@100 metric and 13.26% in map@100 metric. The validity and rationality of the model are proved.

A semi‐supervised segmentation network fusing pseudo‐label with multi‐level feature consistency correction for hard exudates

AbstractTimely detection of hard exudates in fundus images can effectively avoid the severity of the disease, but the labelling of small and numerous lesion areas requires a lot of labour costs. This paper proposes a semi‐supervised segmentation network, which integrates pseudo‐labels and multi‐level features consistency correction. It achieves accurate segmentation of hard exudates by making full use of a small amount of labelled data and a large amount of unlabelled data. The network effectively extracts features from the unlabelled data through knowledge transfer of the teacher‐student model, and incorporates a Transformer network for auxiliary training to promote the quality of transfer. In addition, three unsupervised losses are introduced to improve the performance: the perturbation loss improves the robustness of the model to noise by adding different noises to the same input; the multi‐level feature consistency correction loss ensures the consistency of features of the student model at different scales; and the pseudo‐labelling cross‐supervision loss utilizes the generated pseudo‐labels for supervision between CNN and Transformer. By comparing the segmentation results with different proportion of the labelled data, it has better segmentation performance compared to other methods. The proposed methods can totally increase dice by 16.56% and mean intersection over union (MIoU) by 25.11%.

3D skeleton aware driver behavior recognition framework for autonomous driving system

The recognition of the driver’s behaviors inside an autonomous vehicle can effectively address emergency handling in autonomous driving and is crucial for ensuring the driver’s safety. Driver behavior recognition is a challenging task due to factors such as variations, diversities, complexities, and strong interferences in behaviors. In this paper, to realize the application in the autonomous driving scenes, a novel 3D skeleton aware behavior recognition framework is proposed to recognize various driver behaviors in autonomous driving systems. First, a 3D human pose estimation network (Pose-GTFNet) with temporal Transformer and spatial graph convolutional network (GCN) is designed to infer 3D human poses from 2D pose sequences. Second, based on the obtained 3D human pose sequences, a behavior recognition network (Beh-MSFNet) with multi-skeleton feature fusion is designed to recognize driver behaviors. In the experiments, the Pose-GTFNet and Beh-MSFNet can get the best performance compared with most state-of-the-art (SOTA) methods on the Human3.6M human pose dataset, JHMDB and SHREC action recognition dataset, respectively. In addition, the proposed driver behavior recognition framework can achieve SOTA performance on the Drive&Act and Driver-Skeleton driver behavior datasets.

Aircraft engine fault diagnosis based on fused convolutional Transformer

Aircraft engines will face corrosion and erosion problems when working in harsh gas path environments for a long time, and the fault parameter characteristics are not obvious. Therefore, accurate aircraft engine fault diagnosis methods are of great significance to ensure the safe operation of aircraft. In order to improve the prediction accuracy, an aircraft engine fault diagnosis method based on fused convolutional Transformer is proposed. The self-attention mechanism is used to extract useful features and suppress redundant information. The maximum pooling layer (MaxPool) is introduced into the Transformer model to further reduce the model memory consumption and parameter quantity, and alleviate the overfitting phenomenon. The simulation data set of turbofan engines based on GasTurb modeling is used for verification. The results show that compared with the Transformer network and other traditional deep learning models Back Propagation Neural Networks (BP network), Convolutional Neural Networks (CNN) and Recurrent Neural Network (RNN), the accuracy rates are improved by 6.552%、28.117%、13.189%and respectively 10.29%, which proves the effectiveness of the proposed method and can provide a certain reference for aircraft engine fault diagnosis.

A Method for Single-Phase Ground Fault Section Location in Distribution Networks Based on Improved Empirical Wavelet Transform and Graph Isomorphic Networks

When single-phase ground faults occur in distribution systems, the fault characteristics of zero-sequence current signals are not prominent. They are quickly submerged in noise, leading to difficulties in fault section location. This paper proposes a method for fault section location in distribution networks based on improved empirical wavelet transform (IEWT) and GINs to address this issue. Firstly, based on kurtosis, EWT is optimized using the N-point search method to decompose the zero-sequence current signal into modal components. Noise is filtered out through weighted permutation entropy (WPE), and signal reconstruction is performed to obtain the denoised zero-sequence current signal. Subsequently, GINs are employed for graph classification tasks. According to the topology of the distribution network, the corresponding graph is constructed as the input to the GIN. The denoised zero-sequence current signal is the node input for the GIN. The GIN autonomously explores the features of each graph structure to achieve fault section location. The experimental results demonstrate that this method has strong noise resistance, with a fault section location accuracy of up to 99.95%, effectively completing fault section location in distribution networks.

MDH-Net: advancing 3D brain MRI registration with multi-stage transformer and dual-stream feature refinement hybrid network

MDH-Net: advancing 3D brain MRI registration with multi-stage transformer and dual-stream feature refinement hybrid network

Dynamic deep graph convolution with enhanced transformer networks for time series anomaly detection in IoT

Dynamic deep graph convolution with enhanced transformer networks for time series anomaly detection in IoT

Stable successive Neural Image Compression via coherent demodulation-based transformation

Neural Image Compression (NIC) has made significant strides in recent years. However, the existing NIC methods demonstrate instability issues during iterative re-compression cycles, which can degrade image quality with each cycle. This paper introduces a novel framework aimed at enhancing the stability of NIC methods. We first conducted a theoretical analysis and identified that the instability in current NIC methods stems from a lack of idempotency in transformations. Drawing from the domain of signal processing, we then examined the principles of idempotency in coherent demodulation techniques. This examination led to the identification of three foundational principles that inform the design of stable transformations: the cosine function, parameter sharing, and low-pass filtering. Leveraging these insights, we propose the innovative Coherent Demodulation-based Transformation (CDT), which is designed to address the stability challenges in NIC by incorporating these principles into its architecture. The experimental results suggest that CDT not only significantly improve the re-compression stability but also preserves the codec’s rate–distortion performance. Furthermore, it can be broadly applied in current NIC structures. The effectiveness of the module endorses the viability of designing transformation networks based on Coherent Demodulation principles, playing a crucial role in enhancing stability of NIC. The code will be available at https://github.com/baoyu2020/Stable_SuccessiveNIC.

Efficient memristor accelerator for transformer self-attention functionality

The adoption of transformer networks has experienced a notable surge in various AI applications. However, the increased computational complexity, stemming primarily from the self-attention mechanism, parallels the manner in which convolution operations constrain the capabilities and speed of convolutional neural networks (CNNs). The self-attention algorithm, specifically the matrix-matrix multiplication (MatMul) operations, demands a substantial amount of memory and computational complexity, thereby restricting the overall performance of the transformer. This paper introduces an efficient hardware accelerator for the transformer network, leveraging memristor-based in-memory computing. The design targets the memory bottleneck associated with MatMul operations in the self-attention process, utilizing approximate analog computation and the highly parallel computations facilitated by the memristor crossbar architecture. Remarkably, this approach resulted in a reduction of approximately 10 times in the number of multiply-accumulate (MAC) operations in transformer networks, while maintaining 95.47% accuracy for the MNIST dataset, as validated by a comprehensive circuit simulator employing NeuroSim 3.0. Simulation outcomes indicate an area utilization of 6895.7 μm2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu m^2$$\\end{document}, a latency of 15.52 seconds, an energy consumption of 3 mJ, and a leakage power of 59.55 μW\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mu W$$\\end{document}. The methodology outlined in this paper represents a substantial stride towards a hardware-friendly transformer architecture for edge devices, poised to achieve real-time performance.

An investigation on energy-saving scheduling algorithm of wireless monitoring sensors in oil and gas pipeline networks

With the rapid development of the oil and gas industry, monitoring the safety and efficiency of pipeline networks has become particularly important. In this context, Wireless Sensor Networks (WSNs) are widely used for monitoring oil and gas pipelines due to their flexible deployment and cost-effectiveness. However, since sensor nodes typically rely on limited battery power, extending the network’s lifecycle and improving energy utilization efficiency have become focal points of research. Therefore, this paper proposes an energy-saving scheduling algorithm based on transformer networks, aimed at optimizing energy consumption and data transmission efficiency of wireless monitoring sensors in oil and gas pipelines. Firstly, this study designs a deep learning-based Transformer model that learns from historical data on energy consumption patterns and environmental variables to predict the energy and data transmission needs of each sensor node. Secondly, based on the prediction results, this algorithm employs a dynamic scheduling strategy that automatically adjusts the sensor’s operational mode and communication frequency according to the node’s energy status and task urgency. Additionally, we have validated the effectiveness of the proposed algorithm through field tests and simulation experiments. According to the experimental results, our model has higher efficiency in energy saving. Compared with Convolutional Neural Networks, Recurrent Neural Networks and Graph Neural Networks, the total energy consumption of sensor networks under the model scheduling in this paper was reduced by 6.7%, 33.4% and 26.3%, respectively. Our algorithms improve the energy efficiency and stability of the monitoring system and provide important technical support for future intelligent pipeline monitoring systems. We hope this paper will inspire future scientific research in this field.

Enhanced Infant Movement Analysis Using Transformer-Based Fusion of Diverse Video Features for Neurodevelopmental Monitoring

Neurodevelopment is a highly intricate process, and early detection of abnormalities is critical for optimizing outcomes through timely intervention. Accurate and cost-effective diagnostic methods for neurological disorders, particularly in infants, remain a significant challenge due to the heterogeneity of data and the variability in neurodevelopmental conditions. This study recruited twelve parent–infant pairs, with infants aged 3 to 12 months. Approximately 25 min of 2D video footage was captured, documenting natural play interactions between the infants and toys. We developed a novel, open-source method to classify and analyse infant movement patterns using deep learning techniques, specifically employing a transformer-based fusion model that integrates multiple video features within a unified deep neural network. This approach significantly outperforms traditional methods reliant on individual video features, achieving an accuracy of over 90%. Furthermore, a sensitivity analysis revealed that the pose estimation contributed far less to the model’s output than the pre-trained transformer and convolutional neural network (CNN) components, providing key insights into the relative importance of different feature sets. By providing a more robust, accurate and low-cost analysis of movement patterns, our work aims to enhance the early detection and potential prediction of neurodevelopmental delays, whilst providing insight into the functioning of the transformer-based fusion models of diverse video features.

Nutritional composition analysis in food images: an innovative Swin Transformer approach.

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Early Cervical Cancer Diagnosis with SWIN-Transformer and Convolutional Neural Networks.

Introduction: Early diagnosis of cervical cancer at the precancerous stage is critical for effective treatment and improved patient outcomes. Objective: This study aims to explore the use of SWIN Transformer and Convolutional Neural Network (CNN) hybrid models combined with transfer learning to classify precancerous colposcopy images. Methods: Out of 913 images from 200 cases obtained from the Colposcopy Image Bank of the International Agency for Research on Cancer, 898 met quality standards and were classified as normal, precancerous, or cancerous based on colposcopy and histopathological findings. The cases corresponding to the 360 precancerous images, along with an equal number of normal cases, were divided into a 70/30 train-test split. The SWIN Transformer and CNN hybrid model combines the advantages of local feature extraction by CNNs with the global context modeling by SWIN Transformers, resulting in superior classification performance and a more automated process. The hybrid model approach involves enhancing image quality through preprocessing, extracting local features with CNNs, capturing the global context with the SWIN Transformer, integrating these features for classification, and refining the training process by tuning hyperparameters. Results: The trained model achieved the following classification performances on fivefold cross-validation data: a 94% Area Under the Curve (AUC), an 88% F1 score, and 87% accuracy. On two completely independent test sets, which were never seen by the model during training, the model achieved an 80% AUC, a 75% F1 score, and 75% accuracy on the first test set (precancerous vs. normal) and an 82% AUC, a 78% F1 score, and 75% accuracy on the second test set (cancer vs. normal). Conclusions: These high-performance metrics demonstrate the models' effectiveness in distinguishing precancerous from normal colposcopy images, even with modest datasets, limited data augmentation, and the smaller effect size of precancerous images compared to malignant lesions. The findings suggest that these techniques can significantly aid in the early detection of cervical cancer at the precancerous stage.