Fusion Ability Research Articles

Bearing defect detection based on the improved YOLOv5 algorithm.

In the field of bearing defect detection, Aiming at the problem of low efficiency in manual inspection and prone to missed detections in scenarios with small target defects, and overlapping targets, an improved YOLOv5-based object detection method is proposed. Firstly, in terms of feature extraction, the C3 modules in the original backbone of YOLOv5 are replaced with the finer-grained Res2Block modules to improve the model's feature extraction ability. Secondly, in terms of feature fusion, a Bidirectional Feature Pyramid Network (BiFPN) is added to the original neck of YOLOv5 to enhance the fusion ability of shallow graphic features and deep semantic features. Finally, the performance of the improved YOLOv5 algorithm is validated through ablation experiments and comparative experiments with other defect detection algorithms, including the Small_obj algorithm the existing method of adding a small target detection head for identifying small target defects. The experimental results demonstrate that the improved YOLOv5 algorithm exhibits high mAP and accuracy in bearing defect detection, enabling more precise identification the types of small target defects on bearings in complex scenarios with multiple coexisting defects and overlapping detection targets, thereby providing valuable reference for practical bearing defect detection.

Utilization of transformer model in multimodal data fusion learning: Cross-modal knowledge transfer in the new generation learning space

In response to the difficulties in integrating multimodal data and insufficient model generalization ability in traditional cross-modal knowledge transfer, this article used the Transformer model to explore it in the new generation learning space. Firstly, the article analyzed the processing methods of data and models in cross-modal knowledge transfer, and explored the application of Transformer models in the learning space. This model used natural language processing to represent and extract textual features, Mel Frequency Cepstral Coefficients (MFCCs) to represent and extract audio features, and Faster R-CNN (Faster Region-based Convolutional Neural Network) to represent and extract image features. The article also discussed the implementation process of the Transformer model functionality. The experiment used data from four datasets, including Quora Question Pairs, to test the performance of the model’s cross-modal knowledge transfer through intelligent question answering and task analysis. In single type data testing, the accuracy and recall of the model in this article were better than the comparison model in the three types of data. The highest accuracy and recall in the test set were 91% and 93%, respectively. In the most challenging multimodal intelligent question answering test, the speech-image question answering method achieved an accuracy rate of 89% in answering open questions, indicating that the model had good multimodal data fusion ability. In the analysis experiment of 6 homework prone knowledge points on images with text annotations, the induction accuracy reached 85%, indicating that the model had strong generalization ability. The experimental results showed that the Transformer model had good cross-modal knowledge transfer performance, providing a reference for subsequent research on cross-modal knowledge transfer in the new generation learning space.

A novel learning-based robust model predictive control strategy and case study for application in optimal control of FCEVs

With the gradual implementation of carbon-neutral policies, fuel cell electric vehicles (FCEVs) have gained widespread recognition as a promising solution. Due to multi-source electromechanical coupling, the powertrain of FCEVs constitutes a complex nonlinear system, necessitating an advanced energy management strategy for effective control to enhance economic performance. In this study, a novel learning-based robust model predictive control (LRMPC) is proposed for a 4WD FCEV, integrating a high-confidence velocity estimation model and an accurate state observation model to enhance energy-saving potential, robustness, and real-time application performance under various driving conditions. To enhance robustness and adaptability, deep forest with superior feature fusion abilities is employed to generate future velocity reference datasets based on FCEV operating scenarios. To improve state observation accuracy and control effectiveness, a novel built-in state observation model based on machine learning algorithms is established, leveraging knowledge of nonlinear systems within the FCEV powertrain. Subsequently, to enhance the real-time applicability of LRMPC, an explicit control scheme for multidimensional mapping based on explicit data tables is utilized for accurate state transformation, incorporating input features such as vehicle states and component states. Simulation evaluations and hardware-in-the-loop tests demonstrate the improved economic performance and real-time application ability of LRMPC, showcasing its promising performance.

Improve the Performance of SAR Ship Detectors by Small Object Detection Strategies

Although advanced deep learning techniques have significantly improved SAR ship detection, accurately detecting small ships remains challenging due to their limited size and the few appearance and geometric clues available. In order to solve this problem, we propose several small object detection strategies. The backbone network uses space-to-depth convolution to replace stride and pooling. It reduces information loss during down-sampling. The neck integrates multiple layers of features globally and injects global and local information into different levels. It avoids the inherent information loss of traditional feature pyramid networks and strengthens the information fusion ability without significantly increasing latency. The proposed intersection-of-union considers the center distance and scale of small ships specifically. It reduces the sensitivity of intersection-of-union to positional deviations of small ships, which is helpful for training toward small ships. During training, the smaller the localization loss of small ships, the greater their localization loss gains are. By this, the supervision of small ships is strengthened in the loss function, which can make the losses more biased toward small ships. A series of experiments are conducted on two commonly used datasets, SSDD and SAR-Ship-Dataset. The experimental results show that the proposed method can detect small ships successfully and thus improve the overall performance of detectors.

Interaction mechanism of cold atmospheric plasmas and fusion peptides of spike protein in SARS-CoV-2 revealed by reactive molecular dynamics simulation

Despite the conclusion of the COVID-19 pandemic, the coronavirus-killing ability of cold atmospheric plasma (CAP) remains impressive, and investigations into its underlying mechanisms are still ongoing. The fusion peptide (FP) is a crucial site of membrane fusion and toxicity exerted by the S-protein in severe acute respiratory syndrome CoV-2 (SARS-CoV-2). In this study, reactive molecular dynamics simulations were performed to investigate the interaction mechanisms of FP and reactive oxygen species (ROS). The simulation results show that the given ROS (O atoms and OH radicals as examples) can destroy hydrophobic residues, negatively charged acidic residues, and peptide bonds through structurally altering essential sites. Furthermore, the reaction typically initiates from the H-abstraction reaction, followed by various types of oxidative modifications such as dehydrogenation, hydroxylation, carbonylation, cyclogenesis, ring cleavage, and decarboxylation, which are consistent with the experimental findings made on peptides. Therefore, it can be predicted that the membrane fusion ability of FP and the toxicity of SARS-CoV-2 will be reduced, with CAP functioning as a bactericidal disinfectant. The dose effects were also investigated, providing experimental guidance for the optimization of CAP. In this study, the interaction processes of FP and CAP are explored by revealing the chemical pathways and final reaction products from the computational data, thus providing a fundamental understanding of the mechanisms for inactivating SARS-CoV-2 by CAP.

An Intelligent Fishery Detection Method Based on Cross-Domain Image Feature Fusion

Target detection technology plays a crucial role in fishery ecological monitoring, fishery diversity research, and intelligent aquaculture. Deep learning, with its distinct advantages, provides significant convenience to the fishery industry. However, it still faces various challenges in practical applications, such as significant differences in image species and image blurring. To address these issues, this study proposes a multi-scale, multi-level, and multi-stage cross-domain feature fusion model. In order to train the model more effectively, a new data set called Fish52 (multi-scene fish data set, a data set containing 52 fish species) was constructed, on which the model achieved an mAP (mean average precision is a key measure of model performance) of 82.57%. Furthermore, we compared prevalent one-stage and two-stage detection methods on the Lahatan (single-scene fish data set) and Fish30 data set (a data set containing 30 fish species) and tested them on the F4k (Fish4Knowledge (F4K) is a data set focused on fish detection and identification) and FishNet data set (it is a data set containing 94,532 images from 17,357 aquatic species). The mAP of our proposed model on the Fish30, Lahatan, F4k, and FishNet data sets reaches 91.72%, 98.7%, 88.6%, and 81.5%, respectively, outperforming existing mainstream models. Comprehensive empirical analysis indicates that our model possesses a high generalization ability and reaches advanced performance levels. In this study, the depth of the model backbone is deepened, a novel neck structure is proposed, and a new module is embedded therein. To enhance the fusion ability of the model, a new attention mechanism module is introduced. In addition, in the adaptive decoupling detection head module, introducing classes with independent parameters and regression adapters reduces interaction between different tasks. The proposed model can better monitor fishery resources and enhance aquaculture efficiency. It not only provides an effective approach for fish detection but also has certain reference significance for the identification of similar targets in other environments and offers assistance for the construction of smart fisheries and digital fisheries.

WBi-YOLOSF: improved feature pyramid network for aquatic real-time target detection under the artificial rabbits optimization

As the pillar industry of coastal areas, aquaculture needs artificial intelligence technology to promote economic development. To realize the automation of the aquaculture industry, this paper proposes a new underwater object detection network: WBi-YOLOSF. It realizes the automatic classification and detection of aquatic products, improves the production efficiency of the aquaculture industry, and promotes its economic development. This paper creates an image dataset containing 15 aquatic products to lay the data foundation for model training. In the data preprocessing part, an underwater image enhancement algorithm is proposed to improve the quality of the data set effectively. Aiming at the problem of high false detection rate and missed detection rate of underwater dense small targets, a new data enhancement method was proposed to improve the training set’s data quality comprehensively. Inspired by the weighted bidirectional feature pyramid network, this paper proposes a new feature extraction network: AU-BiFPN, which solves the gradient problem caused by the network hierarchy’s deepening on enhancing the network’s multi-scale feature fusion. The AU-BiFPN network structure is embedded into the YOLO series network framework, significantly improving the basic network’s feature extraction and feature fusion ability and dramatically improving the prediction accuracy without affecting the network inference speed. Here, the swarm intelligence algorithm is introduced to optimize the model hyperparameters, accelerating the convergence speed of model training and significantly reducing the computational cost. At the same time, the model’s accuracy is improved by a cliff. In addition, the Funnel Activation is introduced in the network’s backbone, and the simple, parameter-free attention module is integrated, effectively improving the accuracy and speed of the model prediction. Ablation and comparison experiments show the effectiveness and superiority of the proposed model. Verified by the mean average precision and frame rate evaluation indicators, the experimental results of the WBi-YOLOSF target detection network can reach 0.982 and 203 frames per second, which are 1.4% and five frames per second higher than the network with the second score. In summary, this method can quickly and accurately identify aquatic products, realize real-time target detection of aquatic products, and lay the foundation for developing an aquaculture automation system.

An improved smoking behavior detection algorithm via incorporating an interference information filtering network

Efficient and accurate identification of smoking behavior in public places is crucial for ensuring public health and safety. However, due to various factors like small target size, complex image background, varying cigarette angles and numerous similar objects, current methods still grapple with challenges such as missed detection and false detection when identifying cigarette targets in smoking behavior. This paper presents an enhanced version of the You Only Look Once version5-small algorithm to address these issues effectively. Firstly, to bolster the model's ability for feature extraction of cigarette targets, the Swin Transformer Block structure is integrated into the backbone network to capture long-range dependencies. Secondly, a novel Hybrid Spatial Pyramid Pooling-Fast with Cross Stage Partial Connection module is built based on the foundation of the Spatial Pyramid Pooling-Fast module by integrating both maximum pooling and average pooling to enhance the fusion ability of the multi-scale feature maps. Thirdly, a novel interference information filtering network is introduced to effectively reduce the impact of noise and confusion caused by similar objects, thus enhancing the performance of the model. According to the experimental results, the accuracy of the improved algorithm on the self-made cigarette target image data set reaches 93.5 %, and the recall reaches 89.1 %.

DFW-YOLO: YOLOv5-based algorithm using phased array ultrasonic testing for weld defect recognition

ABSTRACT Although phased array ultrasonic testing (PAUT) technology has been widely employed in pipeline weld defect recognition, current defect recognition still relies on manual operation, which limits its efficiency. By combining deep learning with PAUT, this paper presents a new neural architecture based on YOLOv5 called DFW-YOLO that can call defect indications automatically. To address redundant defect feature maps, the FasterNet backbone is introduced to enhance the model’s feature extraction ability. Additionally, partial dilated convolution (PDConv) and partial convolution (PConv) with an atrous convolution branch are designed to extend the receptive field of the backbone network, mitigating the chance of feature loss. Furthermore, a weighted generalised feature pyramid network (WGFPN) structure is proposed as the neck network. This structure combines the reparameterized generalised feature pyramid network (RepGFPN) with the weighted aggregated concatenation (WAC) structure and eliminates inefficient branches to enhance the model’s feature fusion ability. The experimental results demonstrated that DFW-YOLO outperforms the original YOLOv5 in terms of detection performance. The precision, recall, mean average precision (mAP)@0.5, and mAP@0.5:0.95 are indicators for DFW-YOLO and reach 97.4%, 97.5%, 98.3%, and 68.2%, respectively, marking 4.8%, 5%, 2.7%, and 22.5% improvement compared to the original YOLOv5.

An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys

Marine anchor buoys, as fixed-point profile observation platforms, are highly susceptible to the threat of ship collisions. Installing cameras on buoys can effectively monitor and collect evidence from ships. However, when using a camera to capture images, it is often affected by the continuous shaking of buoys and rainy and foggy weather, resulting in problems such as blurred images and rain and fog occlusion. To address these problems, this paper proposes an improved YOLOv8 algorithm. Firstly, the polarized self-attention (PSA) mechanism is introduced to preserve the high-resolution features of the original deep convolutional neural network and solve the problem of image spatial resolution degradation caused by shaking. Secondly, by introducing the multi-head self-attention (MHSA) mechanism in the neck network, the interference of rain and fog background is weakened, and the feature fusion ability of the network is improved. Finally, in the head network, this model combines additional small object detection heads to improve the accuracy of small object detection. Additionally, to enhance the algorithm’s adaptability to camera detection scenarios, this paper simulates scenarios, including shaking blur, rain, and foggy conditions. In the end, numerous comparative experiments on a self-made dataset show that the algorithm proposed in this study achieved 94.2% mAP50 and 73.2% mAP50:95 in various complex environments, which is superior to other advanced object detection algorithms.

Platelet-Rich Plasma Promotes the Expansion of Human Myoblasts and Favors the In Vitro Generation of Human Muscle Reserve Cells in a Deeper State of Quiescence

Stem cell therapy holds significant potential for skeletal muscle repair, with in vitro-generated human muscle reserve cells (MuRCs) emerging as a source of quiescent myogenic stem cells that can be injected to enhance muscle regeneration. However, the clinical translation of such therapies is hampered by the need for fetal bovine serum (FBS) during the in vitro generation of human MuRCs. This study aimed to determine whether fresh allogeneic human platelet-rich plasma (PRP) combined or not with hyaluronic acid (PRP-HA) could effectively replace xenogeneic FBS for the ex vivo expansion and differentiation of human primary myoblasts. Cells were cultured in media supplemented with either PRP or PRP-HA and their proliferation rate, cytotoxicity and myogenic differentiation potential were compared with those cultured in media supplemented with FBS. The results showed similar proliferation rates among human myoblasts cultured in PRP, PRP-HA or FBS supplemented media, with no cytotoxic effects. Human myoblasts cultured in PRP or PRP-HA showed reduced fusion ability upon differentiation. Nevertheless, we also observed that human MuRCs generated from PRP or PRP-HA myogenic cultures, exhibited increased Pax7 expression and delayed re-entry into the cell cycle upon reactivation, indicating a deeper quiescent state of human MuRCs. These results suggest that allogeneic human PRP effectively replaces FBS for the ex vivo expansion and differentiation of human myoblasts and favors the in vitro generation of Pax7High human MuRCs, with important implications for the advancement of stem cell-based muscle repair strategies.Graphical

A Detection Algorithm for Citrus Huanglongbing Disease Based on an Improved YOLOv8n.

Given the severe impact of Citrus Huanglongbing on orchard production, accurate detection of the disease is crucial in orchard management. In the natural environments, due to factors such as varying light intensities, mutual occlusion of citrus leaves, the extremely small size of Huanglongbing leaves, and the high similarity between Huanglongbing and other citrus diseases, there remains an issue of low detection accuracy when using existing mainstream object detection models for the detection of citrus Huanglongbing. To address this issue, we propose YOLO-EAF (You Only Look Once-Efficient Asymptotic Fusion), an improved model based on YOLOv8n. Firstly, the Efficient Multi-Scale Attention Module with cross-spatial learning (EMA) is integrated into the backbone feature extraction network to enhance the feature extraction and integration capabilities of the model. Secondly, the adaptive spatial feature fusion (ASFF) module is used to enhance the feature fusion ability of different levels of the model so as to improve the generalization ability of the model. Finally, the focal and efficient intersection over union (Focal-EIOU) is utilized as the loss function, which accelerates the convergence process of the model and improves the regression precision and robustness of the model. In order to verify the performance of the YOLO-EAF method, we tested it on the self-built citrus Huanglongbing image dataset. The experimental results showed that YOLO-EAF achieved an 8.4% higher precision than YOLOv8n on the self-built dataset, reaching 82.7%. The F1-score increased by 3.33% to 77.83%, and the mAP (0.5) increased by 3.3% to 84.7%. Through experimental comparisons, the YOLO-EAF model proposed in this paper offers a new technical route for the monitoring and management of Huanglongbing in smart orange orchards.

ReLAP-Net: Residual Learning and Attention Based Parallel Network for Hyperspectral and Multispectral Image Fusion

Remote sensing applications require high-resolution images to obtain precise information about the Earth???s surface. Multispectral images have high spatial resolution but low spectral resolution. Hyperspectral images have high spectral resolution but low spatial resolution. This study proposes a residual learning and attention-based parallel network based on residual network and channel attention. The network performs image fusion of a high spatial resolution multispectral image and a low spatial resolution hyperspectral image. The network training and fusion experiments are conducted on four public benchmark data sets to show the effectiveness of the proposed model. The fusion performance is compared with classical signal processing???based image fusion techniques. Four image metrics are used for the quantitative evaluation of the fused images. The proposed network improved fusion ability by reducing the root mean square error and relative dimensionless global error in synthesis and increased the peak signal-to-noise ratio when compared to other state-of-the-art models.

Design of unattended gas pumping station system based on multi-source data fusion

Abstract Aiming at the problems that some data has been built and distributed in different subsystems during the transformation of the unattended system of an underground gas pumping pump station in Shanxi Tiandi Wangpo Coal industry, such as low data sharing and comprehensive utilization rate due to data isolation, and weak linkage execution ability, a method combining site-level data fusion and platform-level data fusion is proposed. The advantages and disadvantages of the fusion method are analyzed, and the PLC program and WinCC program are written for the corresponding fusion method. The practical application of the underground gas pumping pump station shows that the system has strong fusion ability, which can meet the integration of extraction parameters, environmental parameters, electric parameters, pipeline parameters, working conditions parameters, and other data required by the unattended system, improve the data utilization rate of the unattended system, and realize the complex control functions of the gas pumping system such as one-key start-stop and one-key rotation vacuum pump. The use of data-driven execution instead of manual operation effectively reduces the duty personnel of the underground gas pumping station, provides a guarantee for the less manned or unmanned operation of the gas pumping pump station, reduces the unplanned downtime of the pumping system, improves the efficiency of gas pumping, and ensures the safety of coal mine production.

Detailed analysis of low temperature inactivation of respiratory syncytial virus

Our previous findings indicated that many respiratory syncytial virus (RSV) isolates are unstable at 4 °C compared to 20 °C. Some of the strains completely lose infectivity after 24 h at 4 °C. This study analyzed the inactivation process at 4 °C using a representative strain, RSV/Sendai/851/13. After 24 h of storage at 4 °C, the virus was completely inactivated but retained its ability to attach to and to be taken into host cells. It suggested a reduced fusion ability between the viral and cellular membranes. During storage at 4 °C, the RSV fusion (F) protein underwent a conformational change and was no longer recognized by pre-fusion form-specific antibodies. When the RSV/Sendai/851/13 strain was passaged at 4 °C, a variant with an amino acid substitution, I148T, in the F protein fusion peptide was selected. Also, an amino acid change in G protein demonstrating stability at low temperatures was obtained. These results show that the inactivation of RSV at 4 °C is due to the loss of membrane fusion activity in the F protein, which cannot maintain its pre-fusion state at 4 °C.

DDFNet-A: Attention-Based Dual-Branch Feature Decomposition Fusion Network for Infrared and Visible Image Fusion

The fusion of infrared and visible images aims to leverage the strengths of both modalities, thereby generating fused images with enhanced visible perception and discrimination capabilities. However, current image fusion methods frequently treat common features between modalities (modality-commonality) and unique features from each modality (modality-distinctiveness) equally during processing, neglecting their distinct characteristics. Therefore, we propose a DDFNet-A for infrared and visible image fusion. DDFNet-A addresses this limitation by decomposing infrared and visible input images into low-frequency features depicting modality-commonality and high-frequency features representing modality-distinctiveness. The extracted low and high features were then fused using distinct methods. In particular, we propose a hybrid attention block (HAB) to improve high-frequency feature extraction ability and a base feature fusion (BFF) module to enhance low-frequency feature fusion ability. Experiments were conducted on public infrared and visible image fusion datasets MSRS, TNO, and VIFB to validate the performance of the proposed network. DDFNet-A achieved competitive results on three datasets, with EN, MI, VIFF, QAB/F, FMI, and Qs metrics reaching the best performance on the TNO dataset, achieving 7.1217, 2.1620, 0.7739, 0.5426, 0.8129, and 0.9079, respectively. These values are 2.06%, 11.95%, 21.04%, 21.52%, 1.04%, and 0.09% higher than those of the second-best methods, respectively. The experimental results confirm that our DDFNet-A achieves better fusion performance than state-of-the-art (SOTA) methods.

Intelligent Detection of Surface Defects in High-Speed Railway Ballastless Track Based on Self-Attention and Transfer Learning

The detection of ballastless track surface (BTS) defects is a prerequisite for ensuring the safe operation of high-speed railways. Traditional convolutional neural networks fail to fully exploit contextual information and lack global pixel representations. The extensive stacking of convolutions leads deep learning models to play a black-box detection role, lacking interpretability. Due to the current lack of sufficient high-quality surface data for ballastless tracks, it is a severe constraint on the accurate identification of the substructure state in high-speed railways. This paper proposes an intelligent detection method for BTS defects named TrackNet based on self-attention and transfer learning. The method enhances the fusion ability of global features of BTS defects using multihead self-attention. The model’s dependence on extensive defect data is reduced by transferring knowledge from large-scale publicly available datasets. Experimental results demonstrate that compared to advanced Swin Transformer model results, the TrackNet model achieves improvements in average accuracy and F1-score by 5.15% and 5.16%, respectively, on limited test data. The TrackNet model visualizes the decision regions of the model in identifying BTS defects, revealing the black-box recognition mechanism of deep learning models. This research performs engineering applications and provides valuable insights for the multiclass recognition of BTS defects in high-speed railways.

Terminal strip detection and recognition based on improved YOLOv7-tiny and MAH-CRNN+CTC models

For substation secondary circuit terminal strip wiring, low efficiency, less easy fault detection and inspection, and a variety of other issues, this study proposes a text detection and identification model based on improved YOLOv7-tiny and MAH-CRNN+CTC terminal lines. First, the YOLOv7-tiny target detection model is improved by the introduction of the spatially invariant multi-attention mechanism (SimAM) and the weighted bidirectional feature pyramid network (BiFPN). This also improves the feature enhancements and feature fusion ability of the model, balances various scales of characteristic information, and increases the positioning accuracy of the text test box. Then, a multi-head attention hybrid (MAH) mechanism is implemented to optimize the convolutional recurrent neural network with connectionist temporal classification (CRNN+CTC) so that the model could learn data features with larger weights and increase the recognition accuracy of the model. The findings indicate that the enhanced YOLOv7-tiny model achieves 97.39%, 98.62%, and 95.07% of precision, recall, and mean average precision (mAP), respectively, on the detection dataset. The improved MAH-CRNN+CTC model achieves 91.2% character recognition accuracy in the recognition dataset.

Electrochromic and thermal-control broadband stealth device based on AZO/Ag/AZO configuration electrodes

Existing stealth technologies have limitations including single bands and functions, and static and passive applications, so that targets cannot remain stealthy if multi-source detection approaches are used. With the wide application of visible-infrared (Vis-IR) detection and guidance technologies, there is an urgent need to develop advanced techniques for visible smart stealth and Vis-IR compatible stealth. In view of this, an electrochromic and thermal-control broadband stealth device based on AZO/Ag/AZO configuration electrodes was developed. The influencing mechanisms of microstructural characteristics of the conducting layer in the AZO/Ag/AZO configuration on the visible transmission spectra and infrared (IR) reflection spectra were revealed. Furthermore, a conducting layer enabling IR stealth with high visible transmittance was optimized, on which a V2O5 multi-color electrochromic film layer was prepared via self-assembled deposition. In this way, the device integrates multiple optical mechanisms including the visible-induced transmission, IR radiation suppression, and electrochromism to modulate optical properties in multiple bands. A technical route that integrates visible smart stealth and active suppression of IR radiation was established, which is conducive to improving the adaptive fusion ability under a multi-domain background and multi-band compatible optical stealth.

A new ECT image reconstruction algorithm based on Vision transformer (ViT)

Aiming at the problem of low accuracy of ECT image reconstruction, this paper proposes an ECT image reconstruction algorithm based on Vision Transformer (ViT). ViT is a method that applies the Transformer to the field of image classification, which is characterized by strong long-distance dependency learning ability and strong multi-modal fusion ability compared to CNN. This paper fully utilizes the characteristics of ViT to transform the image reconstruction process of ECT into the classification process of ViT. Here, a method is proposed to use the object field distribution as the image classification label. This method converts a two-dimensional image of the object field distribution into a one-dimensional vector, which is the label vector of the image classification. These vectors and their corresponding reconstructed images obtained by the Landweber algorithm form the sample set of ViT. Extracting a large number of flow pattern samples with various shapes and distributions through COMSOL is used as a training set. After training ViT, this network model is used to infer the labels of the predicted flow patterns. After post-processing this label, the corresponding reconstructed image can be obtained. Finally, simulation experiments are conducted, and the experiments results show that the image errors and correlation coefficients of the reconstructed images obtained through the algorithm in this paper are better than those of Tikhonov algorithm, Landweber algorithm and Long Short-Term Memory Network(LSTM). And this algorithm has better resistance to noise interference than Tikhonov algorithm, Landweber algorithm, and LSTM. This also provides a new approach and means for ECT image reconstruction.