Transformer Model Research Articles

Harnessing synthetic data to address fraud in cross-border payments

The sharing of data between financial institutions is widely recognised as a key component in the industry’s efforts to combat fraud. Broader access to multiple sources of financial data is also critical to the development of high-quality fraud detection mechanisms based on artificial intelligence (AI). Given the challenges relating to sharing real financial data across countries and institutions, the use of synthetic data has recently become critical to enabling the exploration of broader data sharing and supporting open collaboration in AI model development. To generate synthetic data that can substitute for real data, computer algorithms closely mimic the key statistical properties of genuine data, while strictly preserving the privacy and sovereignty of the source data. This paper presents the results of an ongoing exploration into the generation of high-utility synthetic datasets of cross-border payment transactions using transformer models and discusses its application to the development of AI-based fraud prevention solutions.

A deep learning model for personalized intra-arterial therapy planning in unresectable hepatocellular carcinoma: a multicenter retrospective study

Unresectable Hepatocellular Carcinoma (uHCC) poses a substantial global health challenge, demanding innovative prognostic and therapeutic planning tools for improved patient management. The predominant treatment strategies include Transarterial chemoembolization (TACE) and hepatic arterial infusion chemotherapy (HAIC). Between January 2014 and November 2021, a total of 1725 uHCC patients [mean age, 52.8±11.5 years; 1529 males] received preoperative CECT scans and were eligible for TACE or HAIC. Patients were assigned to one of the four cohorts according to their treatment, four transformer models (SELECTION) were trained and validated on each cohort; AUC was used to determine the prognostic performance of the trained models. Patients were stratified into high and low-risk groups based on the survival scores computed by SELECTION. The proposed AI-based treatment decision model (ATOM) utilizes survival scores to further inform final therapeutic recommendation. In this study, the training and validation sets included 1448 patients, with an additional 277 patients allocated to the external validation sets. The SELECTION model outperformed both clinical models and the ResNet approach in terms of AUC. Specifically, SELECTION-TACE and SELECTION-HAIC achieved AUCs of 0.761 (95% CI, 0.693-0.820) and 0.805 (95% CI, 0.707-0.881) respectively, in predicting ORR in their external validation cohorts. In predicting OS, SELECTION-TC and SELECTION-HC demonstrated AUCs of 0.736 (95% CI, 0.608-0.841) and 0.748 (95% CI, 0.599-0.865) respectively, in their external validation sets. SELECTION-derived survival scores effectively stratified patients into high and low-risk groups, showing significant differences in survival probabilities (P<0.05 across all four cohorts). Additionally, the concordance between ATOM and clinician recommendations was associated with significantly higher response/survival rates in cases of agreement, particularly within the TACE, HAIC, and TC cohorts in the external validation sets (P<0.05). ATOM was proposed based on SELECTION-derived survival scores, emerges as a promising tool to inform the selection among different intra-arterial interventional therapy techniques. This study received funding from the Beijing Municipal Natural Science Foundation, China (Z190024); the Key Program of the National Natural Science Foundation of China, China (81930119); The Science and Technology Planning Program of Beijing Municipal Science & Technology Commission and Administrative Commission of Zhongguancun Science Park, China (Z231100004823012); Tsinghua University Initiative Scientific Research Program of Precision Medicine, China (10001020108); and Institute for Intelligent Healthcare, Tsinghua University, China (041531001).

An ensemble approach integrating LSTM and ARIMA models for enhanced financial market predictions.

Forecasting financial markets is a complex task that requires addressing various challenges, such as market complexity, data heterogeneity, the need for rapid response and constant changes in conditions, to gain a competitive advantage. To effectively address these challenges, it is necessary to constantly improve existing and develop new methods of intelligent forecasting, which will improve the accuracy of forecasts, reduce risks and increase the productivity of financial decision-making processes. In this article, we study and analyse forecasting methods in financial markets, such as support vector regression (SVR), autoregressive integrated moving average (ARIMA), long short-term memory recurrent neural network (LSTM) and extreme gradient boosting algorithm (XG-Boost). Based on this analysis, we propose an ensemble forecasting procedure that integrates LSTM and ARIMA models. Due to the careful combination of these models, our approach yields better results than individual methods. For example, our model demonstrates a significant 15% improvement in root mean square error (RMSE) and a slight improvement in coefficient of determination compared with LSTM. Furthermore, simulation results obtained on three real-world datasets and evaluated using the RMSE criterion confirm the superiority of our proposed method over alternative methods such as LSTMs, transformer models and optimized deep recurrent neural networks with long short-term memory for financial market forecasting. Furthermore, our approach creates the prerequisites for parallelizing both models, thus providing an opportunity to accelerate forecasting results in future research.

UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation.

Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks. Within the transformer models, the self-attention mechanism is one of the main building blocks that strives to capture long-range dependencies, compared to the local convolutional-based design. However, the self-attention operation has quadratic complexity which proves to be a computational bottleneck, especially in volumetric medical imaging, where the inputs are 3D with numerous slices. In this paper, we propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features using a pair of inter-dependent branches based on spatial and channel attention. Our spatial attention formulation is efficient and has linear complexity with respect to the input. To enable communication between spatial and channel-focused branches, we share the weights of query and key mapping functions that provide a complimentary benefit (paired attention), while also reducing the complexity. Our extensive evaluations on five benchmarks, Synapse, BTCV, ACDC, BraTS, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy. On Synapse, our UNETR++ sets a new state-of-the-art with a Dice Score of 87.2%, while significantly reducing parameters and FLOPs by over 71%, compared to the best method in the literature. Our code and models are available at: https://tinyurl.com/2p87x5xn.

Harmony search for hyperparameters optimization of a low resource language transformer model trained with a novel parallel corpus Ocelotl Nahuatl – Spanish

Nahuatl, a low-resource language, does not have an online translator application. Instead, resources are limited to dictionaries, web pages, or digital books. Given this condition, it is vital to provide as much support to the language as possible. This research aims to enhance the BLEU score in machine translation by applying the harmony search heuristic method to state-of-the-art transformers models. This is conducted by finding the optimal hyperparameter settings for the models. Models are trained and tested using a fresh moderate-size parallel corpus of 1.5k phrases. By utilizing harmony search, the study shows an improvement in the BLEU score, enhancing it by 2.569%. In order to accomplish this, various factors related to the hyperparameters need to be considered. The application of harmony search with transformers can be extended to various parallel corpora or models, taking these considerations into account.

ATAT: Astronomical Transformer for time series and Tabular data

Context. The advent of next-generation survey instruments, such as the Vera C. Rubin Observatory and its Legacy Survey of Space and Time (LSST), is opening a window for new research in time-domain astronomy. The Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC) was created to test the capacity of brokers to deal with a simulated LSST stream. Aims. Our aim is to develop a next-generation model for the classification of variable astronomical objects. We describe ATAT, the Astronomical Transformer for time series And Tabular data, a classification model conceived by the ALeRCE alert broker to classify light curves from next-generation alert streams. ATAT was tested in production during the first round of the ELAsTiCC campaigns. Methods. ATAT consists of two transformer models that encode light curves and features using novel time modulation and quantile feature tokenizer mechanisms, respectively. ATAT was trained on different combinations of light curves, metadata, and features calculated over the light curves. We compare ATAT against the current ALeRCE classifier, a balanced hierarchical random forest (BHRF) trained on human-engineered features derived from light curves and metadata. Results. When trained on light curves and metadata, ATAT achieves a macro F1 score of 82.9 ± 0.4 in 20 classes, outperforming the BHRF model trained on 429 features, which achieves a macro F1 score of 79.4 ± 0.1. Conclusions. The use of transformer multimodal architectures, combining light curves and tabular data, opens new possibilities for classifying alerts from a new generation of large etendue telescopes, such as the Vera C. Rubin Observatory, in real-world brokering scenarios.

HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs.

The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ( ∼5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.

Development and external validation of temporal fusion transformer models for continuous intraoperative blood pressure forecasting

During surgery, intraoperative hypotension is associated with postoperative morbidity and should therefore be avoided. Predicting the occurrence of hypotension in advance may allow timely interventions to prevent hypotension. Previous prediction models mostly use high-resolution waveform data, which is often not available. We utilised a novel temporal fusion transformer (TFT) algorithm to predict intraoperative blood pressure trajectories 7min in advance. We trained the model with low-resolution data (sampled every 15s) from 73,009 patients who were undergoing general anaesthesia for non-cardiothoracic surgery between January 1, 2017, and December 30, 2020, at the General Hospital of Vienna, Austria. The data set contained information on patient demographics, vital signs, medication, and ventilation. The model was evaluated using an internal (n=8113) and external test set (n=5065) obtained from the openly accessible Vital Signs Database. In the internal test set, the mean absolute error for predicting mean arterial blood pressure was 0.376 standard deviations-or 4mmHg-and 0.622 standard deviations-or 7mmHg-in the external test set. We also adapted the TFT model to binarily predict the occurrence of hypotension as defined by mean arterial blood pressure < 65mmHg in the next one, three, five, and 7min. Here, model discrimination was excellent, with a mean area under the receiver operating characteristic curve (AUROC) of 0.933 in the internal test set and 0.919 in the external test set. Our TFT model is capable of accurately forecasting intraoperative arterial blood pressure using only low-resolution data showing a low prediction error. When used for binary prediction of hypotension, we obtained excellent performance. No external funding.

Exploring vision transformers for classifying early Barrett's dysplasia in endoscopic images: A pilot study on white-light and narrow-band imaging.

Various deep learning models, based on convolutional neural network (CNN), have been shown to improve the detection of early esophageal neoplasia in Barrett's esophagus. Vision transformer (ViT), derived from natural language processing, has emerged as the new state-of-the-art for image recognition, outperforming predecessors such as CNN. This pilot study explores the use of ViT to classify the presence or absence of early esophageal neoplasia in endoscopic images of Barrett's esophagus. A BO dataset of 1918 images of Barrett's esophagus from 267 unique patients was used. The images were classified as dysplastic (D-BO) or non-dysplastic (ND-BO). A pretrained vision transformer model, ViTBase16, was used to develop our classifier models. Three ViT models were developed for comparison based on imaging modality: white-light imaging (WLI), narrow-band imaging (NBI), and combined modalities. Performance of each model was evaluated based on accuracy, sensitivity, specificity, confusion matrices, and receiver operating characteristic curves. The ViT models demonstrated the following performance: WLI-ViT (Accuracy: 92%, Sensitivity: 82%, Specificity: 95%), NBI-ViT (Accuracy: 99%, Sensitivity: 97%, Specificity: 99%), and combined modalities-ViT (Accuracy: 93%, Sensitivity: 87%, Specificity: 95%). Combined modalities-ViT showed greater accuracy (94% vs 90%) and sensitivity (80% vs 70%) compared with WLI-ViT when classifying WLI images on a subgroup testing set. ViT exhibited high accuracy in classifying the presence or absence of EON in endoscopic images of Barrett's esophagus. ViT has the potential to be widely applicable to other endoscopic diagnoses of gastrointestinal diseases.

An aircraft engine remaining useful life prediction method based on predictive vector angle minimization and feature fusion gate improved transformer model

Remaining useful life (RUL) prediction is crucial for achieving intelligent and predictive maintenance of aircraft engines. In practical applications, advance prediction values smaller than the true values can prevent serious deferred maintenance accidents. Using asymmetric loss functions directly leads to noticeable accuracy degradation, and existing methods often fail to satisfy accuracy and advance prediction requirements. To address this problem, this paper proposes a novel RUL prediction method based on the Prediction Vector Angle (PVA) minimization and Feature Fusion Gate (FFG) improved Transformer network. Specifically, the FFG is proposed to enhance Transformer prediction accuracy by dynamically fusing global and local features. The concept of PVA is first introduced based on the tilting properties of the RUL descent process. The target of the prediction model is cleverly transformed from error minimization to PVA minimization through the cosine similarity loss function. Various experiments on the CMAPSS dataset demonstrate the effectiveness of the proposed method in achieving high accuracy and advanced prediction. Compared to the state-of-the-art method, RMSE is reduced by at least 2.94 % and Score by 7.00 %. Finally, the PVA minimization mechanism significantly improves long short-term memory and convolutional neural network performance. The proposed method is noteworthy for its superiority and applicability.

Improving Clinical Decision Making With a Two-Stage Recommender System.

Clinical decision-making is complex and time-intensive. To help in this effort, clinical recommender systems (RS) have been designed to facilitate healthcare practitioners with personalized advice. However, designing an effective clinical RS poses challenges due to the multifaceted nature of clinical data and the demand for tailored recommendations. In this article, we introduce a 2-Stage Recommendation framework for clinical decision-making, which leverages a publicly accessible dataset of electronic health records. In the first stage, a deep neural network-based model is employed to extract a set of candidate items, such as diagnoses, medications, and prescriptions, from a patient's electronic health records. Subsequently, the second stage utilizes a deep learning model to rank and pinpoint the most relevant items for healthcare providers. Both retriever and ranker are based on pre-trained transformer models that are stacked together as a pipeline. To validate our model, we compared its performance against several baseline models using different evaluation metrics. The results reveal that our proposed model attains a performance gain of approximately 12.3% macro-average F1 compared to the second best performing baseline. Qualitative analysis across various dimensions also confirms the model's high performance. Furthermore, we discuss challenges like data availability, privacy concerns, and shed light on future exploration in this domain.

TransFrameNet: A transformer-based approach for generalized seismic performance prediction of building structures

Performing nonlinear seismic analysis on a large number of building structures is challenging. Deep learning offers rapid prediction but still with limitations. One model is generally only applicable to a specific building structure and not easily extended to others. To address this, TransFrameNet, a new method based on Transformer, is proposed. By converting buildings into archetypes, TransFrameNet is able to consider the variations of different buildings in geometric features and component sizes. With hard parameter sharing, multi-task learning further expands the applications for multiple building structures with different designs. TransFrameNet is tested on 100 steel moment resisting frames (SMRFs) to predict floor displacement response using 40 seismic ground motions. Results reveal that TransFrameNet can accurately predict the displacement response of different buildings, with an average mean squared error of 0.0037. Notably, compared to LSTM and Transformer models, TransFrameNet shows significantly improved correlation when tested on a 20-story SMRF structure.

Evaluating the Effectiveness of Time Series Transformers for Demand Forecasting in Retail

This study investigates the effectiveness of Transformer-based models for retail demand forecasting. We evaluated vanilla Transformer, Informer, Autoformer, PatchTST, and temporal fusion Transformer (TFT) against traditional baselines like AutoARIMA and AutoETS. Model performance was assessed using mean absolute scaled error (MASE) and weighted quantile loss (WQL). The M5 competition dataset, comprising 30,490 time series from 10 stores, served as the evaluation benchmark. The results demonstrate that Transformer-based models significantly outperform traditional baselines, with Transformer, Informer, and TFT leading the performance metrics. These models achieved MASE improvements of 26% to 29% and WQL reductions of up to 34% compared to the seasonal Naïve method, particularly excelling in short-term forecasts. While Autoformer and PatchTST also surpassed traditional methods, their performance was slightly lower, indicating the potential for further tuning. Additionally, this study highlights a trade-off between model complexity and computational efficiency, with Transformer models, though computationally intensive, offering superior forecasting accuracy compared to the significantly slower traditional models like AutoARIMA. These findings underscore the potential of Transformer-based approaches for enhancing retail demand forecasting, provided the computational demands are managed effectively.

Определение факта насыщения магнитопровода трансформатора тока на основе непрерывного измерения сопротивления цепи намагничивания

Saturation of electromagnetic current transformers in transient modes of operation of an electric power system can lead to incorrect operation of relay protection devices. One of the most promising directions to improve the stability of the operation of relay protection devices in such modes is digital correction of secondary current. Currently, effective methods have been developed to restore the secondary current based on determination of magnetic circuit saturation. However, the practical application of these methods is impossible due to the imperfection of existing methods of secondary current segmentation. Thus, it is urgent to develop a method to determine saturation for implementation of digital correction algorithms for electromagnetic current transformers. To solve the stated problem, methods of mathematical modeling of electrical circuits containing transformers and the small-disturbance theory have been used. Verification of the research results is carried out in the Matlab software package. A mathematical criterion is formulated to determine the saturation of the magnetic circuit of an electromagnetic current transformer based on the inductance of the magnetization circuit. A method of segmentation of the secondary current has been developed for the implementation of digital correction algorithms based on the proposed criterion. The effectiveness of the developed method to determine the saturation of the magnetic circuit of an electromagnetic current transformer using computational experiments in the Matlab software package is demonstrated. It is planned to verify the developed method using a physical model of an electromagnetic current transformer.

Enhancing Chinese–Braille translation: A two-part approach with token prediction and segmentation labeling

Visually assistive systems for the visually impaired play a pivotal role in enhancing the quality of life for the visually impaired. Assistive technologies for the visually impaired have undergone a remarkable transformation with the advent of deep learning and sophisticated assistive devices. In particular, the paper utilizes the latest machine translation models and techniques to accomplish the Chinese–Braille translation task, providing convenience for visually impaired individuals. The Traditional end-to-end Chinese–Braille translation approach incorporates Braille dots and Braille word segmentation symbols as tokens within the model’s vocabulary. However, our findings reveal that Braille word segmentation is significantly more complex than Braille dot prediction. The paper proposes a novel Two-Part Loss (TPL) method that treats these tasks distinctly, leading to significant accuracy improvements. To enhance translation performance further, we introduce a BERT-Enhanced Segmentation Transformer (BEST) method. BEST leverages knowledge distillation techniques to transfer knowledge from a pre-trained BERT model to the translate model, mitigating its limitations in word segmentation. Additionally, soft label distillation is employed to improve overall efficacy further. The TPL approach achieves an average BLEU score improvement of 1.16 and 5.42 for Transformer and GPT models on four datasets, respectively. In addition, The work presents a two-stage deep learning-based translation approach that outperforms traditional multi-step and end-to-end methods. The proposed two-stage translation method achieves an average BLEU score improvement of 0.85 across four datasets.

Integrating self-attention mechanisms in deep learning: A novel dual-head ensemble transformer with its application to bearing fault diagnosis

In this paper, we propose a novel dual-head ensemble Transformer (DHET) algorithm for the classification of signals with time–frequency features such as bearing vibration signals. The DHET model employs a dual-input time–frequency architecture, integrating a 1D Transformer model and a 2D Vision Transformer model to capture the spatial and time–frequency features. By utilizing data from both the time and time–frequency domains, the proposed algorithm broadens its feature extraction capabilities and enhances the model’s capacity for generalization. In our DHET structure, the original Transformer model leverages self-attention mechanisms to consider relationships among signal input segmentations, which makes it effective at capturing long-range dependencies in signal data, while the Vision Transformer model takes 2D images as input and creates the image patches for embedding and each patch is linearly embedded into a flat vector and treated as a ‘token,’ then the ‘tokens’ are processed by the Transformer layers to learn global contextual representations, enabling the model to perform signal classification task. This integration notably enhances the performance and capability of the model. Our DHET is especially effective for rolling bearing fault diagnosis. The simulation results show that the proposed DHET has higher classification accuracy for bearing fault diagnosis and outperforms CNN-based methods.

Detection of Harmful H2S Concentration Range, Health Classification, and Lifespan Prediction of CH4 Sensor Arrays in Marine Environments

Underwater methane (CH4) detection technology is of great significance to the leakage monitoring and location of marine natural gas transportation pipelines, the exploration of submarine hydrothermal activity, and the monitoring of submarine volcanic activity. In order to improve the safety of underwater CH4 detection mission, it is necessary to study the effect of hydrogen sulfide (H2S) in leaking CH4 gas on sensor performance and harmful influence, so as to evaluate the health status and life prediction of underwater CH4 sensor arrays. In the process of detecting CH4, the accuracy decreases when H2S is found in the ocean water. In this study, we proposed an explainable sorted-sparse (ESS) transformer model for concentration interval detection under industrial conditions. The time complexity was decreased to O (n logn) using an explainable sorted-sparse block. Additionally, we proposed the Ocean X generative pre-trained transformer (GPT) model to achieve the online monitoring of the health of the sensors. The ESS transformer model was embedded in the Ocean X GPT model. When the program satisfied the special instructions, it would jump between models, and the online-monitoring question-answering session would be completed. The accuracy of the online monitoring of system health is equal to that of the ESS transformer model. This Ocean-X-generated model can provide a lot of expert information about sensor array failures and electronic noses by text and speech alone. This model had an accuracy of 0.99, which was superior to related models, including transformer encoder (0.98) and convolutional neural networks (CNN) + support vector machine (SVM) (0.97). The Ocean X GPT model for offline question-and-answer tasks had a high mean accuracy (0.99), which was superior to the related models, including long short-term memory–auto encoder (LSTM–AE) (0.96) and GPT decoder (0.98).

Drone image recognition and intelligent power distribution network equipment fault detection based on the transformer model and transfer learning

Drone image recognition and intelligent power distribution network equipment fault detection based on the transformer model and transfer learning

Revolutionizing health monitoring: Integrating transformer models with multi-head attention for precise human activity recognition using wearable devices.

A daily activity routine is vital for overall health and well-being, supporting physical and mental fitness. Consistent physical activity is linked to a multitude of benefits for the body, mind, and emotions, playing a key role in raising a healthy lifestyle. The use of wearable devices has become essential in the realm of health and fitness, facilitating the monitoring of daily activities. While convolutional neural networks (CNN) have proven effective, challenges remain in quickly adapting to a variety of activities. This study aimed to develop a model for precise recognition of human activities to revolutionize health monitoring by integrating transformer models with multi-head attention for precise human activity recognition using wearable devices. The Human Activity Recognition (HAR) algorithm uses deep learning to classify human activities using spectrogram data. It uses a pretrained convolution neural network (CNN) with a MobileNetV2 model to extract features, a dense residual transformer network (DRTN), and a multi-head multi-level attention architecture (MH-MLA) to capture time-related patterns. The model then blends information from both layers through an adaptive attention mechanism and uses a SoftMax function to provide classification probabilities for various human activities. The integrated approach, combining pretrained CNN with transformer models to create a thorough and effective system for recognizing human activities from spectrogram data, outperformed these methods in various datasets - HARTH, KU-HAR, and HuGaDB produced accuracies of 92.81%, 97.98%, and 95.32%, respectively. This suggests that the integration of diverse methodologies yields good results in capturing nuanced human activities across different activities. The comparison analysis showed that the integrated system consistently performs better for dynamic human activity recognition datasets. In conclusion, maintaining a routine of daily activities is crucial for overall health and well-being. Regular physical activity contributes substantially to a healthy lifestyle, benefiting both the body and the mind. The integration of wearable devices has simplified the monitoring of daily routines. This research introduces an innovative approach to human activity recognition, combining the CNN model with a dense residual transformer network (DRTN) with multi-head multi-level attention (MH-MLA) within the transformer architecture to enhance its capability.

Fault analysis of oil-immersed transformer based on digital twin technology

Oil-immersed transformer is one of the important equipment to support the safe and stable operation of power system. At the same time, in the process of transformer operation, transformer failure is inevitable, this kind of failure belongs to multiple latent faults, which seriously endangers the safe operation of the transformer. Therefore, the maintenance of the transformer has played a crucial role. A fault detection method based on digital twin model is proposed to solve the problem that the internal dynamic characteristics of oil-immersed transformer are difficult to extract, and the fault detection error is large due to noise interference. The dynamic model of oil-immersed transformer is constructed, the main cause of failure is analysed, the failure time of internal insulator is taken as a linear index, and the linear correlation between failure probability and failure time is solved by Weibull distribution function. The fitting function between transformer fault residence time, operating days and fault probability variance is established, and the fitting value obtained is used as the initial data reference of the fault detection model. The digital twin model is used to establish the dynamic fault probability calculation function, and the on-site data is substituted to calculate the fault detection threshold, and then the data is compared to complete the detection.