Local Transformations Research Articles

LET: a local enhancement transformer for low-light image enhancement

LET: a local enhancement transformer for low-light image enhancement

Local Optimum Time-Reassigned Synchrosqueezing Transform for Bearing Fault Diagnosis of Rotating Equipment

Local Optimum Time-Reassigned Synchrosqueezing Transform for Bearing Fault Diagnosis of Rotating Equipment

Cross-modal credibility modelling for EEG-based multimodal emotion recognition

Objective. The study of emotion recognition through electroencephalography (EEG) has garnered significant attention recently. Integrating EEG with other peripheral physiological signals may greatly enhance performance in emotion recognition. Nonetheless, existing approaches still suffer from two predominant challenges: modality heterogeneity, stemming from the diverse mechanisms across modalities, and fusion credibility, which arises when one or multiple modalities fail to provide highly credible signals. Approach. In this paper, we introduce a novel multimodal physiological signal fusion model that incorporates both intra-inter modality reconstruction and sequential pattern consistency, thereby ensuring a computable and credible EEG-based multimodal emotion recognition. For the modality heterogeneity issue, we first implement a local self-attention transformer to obtain intra-modal features for each respective modality. Subsequently, we devise a pairwise cross-attention transformer to reveal the inter-modal correlations among different modalities, thereby rendering different modalities compatible and diminishing the heterogeneity concern. For the fusion credibility issue, we introduce the concept of sequential pattern consistency to measure whether different modalities evolve in a consistent way. Specifically, we propose to measure the varying trends of different modalities, and compute the inter-modality consistency scores to ascertain fusion credibility. Main results. We conduct extensive experiments on two benchmarked datasets (DEAP and MAHNOB-HCI) with the subject-dependent paradigm. For the DEAP dataset, our method improves the accuracy by 4.58%, and the F1 score by 0.63%, compared to the state-of-the-art baseline. Similarly, for the MAHNOB-HCI dataset, our method improves the accuracy by 3.97%, and the F1 score by 4.21%. In addition, we gain much insight into the proposed framework through significance test, ablation experiments, confusion matrices and hyperparameter analysis. Consequently, we demonstrate the effectiveness of the proposed credibility modelling through statistical analysis and carefully designed experiments. Significance. All experimental results demonstrate the effectiveness of our proposed architecture and indicate that credibility modelling is essential for multimodal emotion recognition.

Twisting asymptotic symmetries and algebraically special vacuum solutions

In this paper, we study asymptotic symmetries and algebraically special exact solutions in the Newman-Penrose formalism. Removing the hypersurface orthogonal condition in the well studied Newman-Unti gauge, we obtain a generic asymptotic solution space which includes all possible origins of propagating degree of freedom. The asymptotic symmetry of the generalized system extends the Weyl-BMS symmetry by two independent local Lorentz transformations with non-trivial boundary charges, which reveals new boundary degrees of freedom. The generalized Newman-Unti gauge includes algebraically special condition in its most convenient form. Remarkably, the generic solutions satisfying the algebraically special condition truncate in the inverse power of radial expansions and the non-radial Newman-Penrose equations are explicitly solved at any order. Hence, we provide the most general algebraically special solution space and the derivation is self-contained in the Newman-Penrose formalism. The asymptotic symmetry with respect to the algebraically special condition is the standard Weyl-BMS symmetry and the symmetry parameters consist only the integration constant order. We present the Kerr solution and Taub-NUT solution in the generalized Newman-Unti gauge in a simple form.

Rotationally Invariant Circuits: Universality with the Exchange Interaction and Two Ancilla Qubits.

Universality of local unitary transformations is one of the cornerstones of quantum computing with many applications and implications that go beyond this field. However, it has recently been shown that this universality does not hold in the presence of continuous symmetries: generic symmetric unitaries on a composite system cannot be implemented, even approximately, using local symmetric unitaries on the subsystems. In this Letter, we show that, despite these constraints, any SU(2) rotationally invariant unitary can be realized with the Heisenberg exchange interaction, which is 2-local and rotationally invariant, provided that the system interacts with a pair of ancilla qubits. We also show that a single ancilla is not enough to achieve universality. Furthermore, we study qubit circuits formed from k-local rotationally invariant unitaries and fully characterize the constraints imposed by locality on the realizable unitaries. We also find an interpretation of these constraints in terms of the average energy of states with a fixed angular momentum.

A dense and U-shaped transformer with dual-domain multi-loss function for sparse-view CT reconstruction.

CT image reconstruction from sparse-view projections is an important imaging configuration for low-dose CT, as it can reduce radiation dose. However, the CT images reconstructed from sparse-view projections by traditional analytic algorithms suffer from severe sparse artifacts. Therefore, it is of great value to develop advanced methods to suppress these artifacts. In this work, we aim to use a deep learning (DL)-based method to suppress sparse artifacts. Inspired by the good performance of DenseNet and Transformer architecture in computer vision tasks, we propose a Dense U-shaped Transformer (D-U-Transformer) to suppress sparse artifacts. This architecture exploits the advantages of densely connected convolutions in capturing local context and Transformer in modelling long-range dependencies, and applies channel attention to fusion features. Moreover, we design a dual-domain multi-loss function with learned weights for the optimization of the model to further improve image quality. Experimental results of our proposed D-U-Transformer yield performance improvements on the well-known Mayo Clinic LDCT dataset over several representative DL-based models in terms of artifact suppression and image feature preservation. Extensive internal ablation experiments demonstrate the effectiveness of the components in the proposed model for sparse-view computed tomography (SVCT) reconstruction. The proposed method can effectively suppress sparse artifacts and achieve high-precision SVCT reconstruction, thus promoting clinical CT scanning towards low-dose radiation and high-quality imaging. The findings of this work can be applied to denoising and artifact removal tasks in CT and other medical images.

Least-Squares Estimation of Tangent Space for a Triangular Mesh

This paper presents a novel method for computing an orthonormal tangent space at every vertex of a triangular mesh provided with texture coordinates, which is essential in normal mapping. Our method employs least-squares estimation of rotation between two sets of corresponding points: one set from the positions of the 1-ring neighbors of a vertex in texture space and the other set from their corresponding positions in world space. This optimal rotation represents a local space transformation that defines the tangent space at the vertex. Consequently, our method ensures that the resulting tangent space is inherently orthonormal, eliminating the need for intricate operations such as averaging and orthonormalization, which are required in the previous methods. In addition, we introduce texture space alignment of triangles to address vertices on seams, which are inevitable in texture mapping to a closed surface mesh. Experimental results demonstrate the effectiveness of the proposed methods in various experiments.

Mask-Homo: Pseudo Plane Mask-Guided Unsupervised Multi-Homography Estimation

Homography estimation is a fundamental problem in computer vision. Previous works mainly focus on estimating either a single homography, or multiple homographies based on mesh grid division of the image. In practical scenarios, single homography is inadequate and often leads to a compromised result for multiple planes; while mesh grid multi-homography damages the plane distribution of the scene, and does not fully address the restriction to use homography. In this work, we propose a novel semantics guided multi-homography estimation framework, Mask-Homo, to provide an explicit solution to the multi-plane depth disparity problem. First, a pseudo plane mask generation module is designed to obtain multiple correlated regions that follow the plane distribution of the scene. Then, multiple local homography transformations, each of which aligns a correlated region precisely, are predicted and corresponding warped images are fused to obtain the final result. Furthermore, a new metric, Mask-PSNR, is proposed for more comprehensive evaluation of alignment. Extensive experiments are conducted to verify the effectiveness of the proposed method. Our code is available at https://github.com/SAITPublic/MaskHomo.

Image rectangling network based on reparameterized transformer and assisted learning

Stitched images can offer a broader field of view, but their boundaries can be irregular and unpleasant. To address this issue, current methods for rectangling images start by distorting local grids multiple times to obtain rectangular images with regular boundaries. However, these methods can result in content distortion and missing boundary information. We have developed an image rectangling solution using the reparameterized transformer structure, focusing on single distortion. Additionally, we have designed an assisted learning network to aid in the process of the image rectangling network. To improve the network’s parallel efficiency, we have introduced a local thin-plate spline Transform strategy to achieve efficient local deformation. Ultimately, the proposed method achieves state-of-the-art performance in stitched image rectangling with a low number of parameters while maintaining high content fidelity. The code is available at https://github.com/MelodYanglc/TransRectangling.

Block Image Compressive Sensing with Local and Global Information Interaction

Block image compressive sensing methods, which divide a single image into small blocks for efficient sampling and reconstruction, have achieved significant success. However, these methods process each block locally and thus disregard the global communication among different blocks in the reconstruction step. Existing methods have attempted to address this issue with local filters or by directly reconstructing the entire image, but they have only achieved insufficient communication among adjacent pixels or bypassed the problem. To directly confront the communication problem among blocks and effectively resolve it, we propose a novel approach called Block Reconstruction with Blocks' Communication Network (BRBCN). BRBCN focuses on both local and global information, while further taking their interactions into account. Specifically, BRBCN comprises dual CNN and Transformer architectures, in which CNN is used to reconstruct each block for powerful local processing and Transformer is used to calculate the global communication among all the blocks. Moreover, we propose a global-to-local module (G2L) and a local-to-global module (L2G) to effectively integrate the representations of CNN and Transformer, with which our BRBCN network realizes the bidirectional interaction between local and global information. Extensive experiments show our BRBCN method outperforms existing state-of-the-art methods by a large margin. The code is available at https://github.com/kongxiuxiu/BRBCN

HRU-TNet: Hybrid Residual U-Transformer Network for diabetic retinopathy multi-lesion segmentation

The automatic segmentation of diabetic retinopathy (DR) holds significant importance for assisting physicians in diagnosis and treatment. Given the complexity, high inter-class similarity, and uncertainty of DR, it is crucial to integrate multiscale information between lesions and establish global correlations among them. To address these issues, a novel HRU-TNet (Hybrid Residual U-Transformer Network) algorithm for retinal lesion segmentation is proposed. In this framework, the network is augmented with lightweight self-attention residual U-modules (LSA-RSU) to capture high-frequency details of the lesions and global contextual information. The skip connections are then enhanced through interactive residual transformer fusion modules (IRTF) and channel-cross attention (CCA), promoting dependencies among features at different scales and filtering out interfering information to guide feature fusion and eliminate ambiguity. Additionally, a novel retinal image enhancement technique is devised, employing local wavelet transformations to capture detailed components of the retinal images, thereby enhancing the representational capacity of the segmentation network. Data augmentation is also performed to ensure network adaptability to small datasets. Comprehensive experiments conducted on the publicly available IDRID and e_ophtha datasets yielded average AUC_PR values of 0.709 and 0.451, respectively. The proposed approach demonstrated superior generalization on the DDR dataset compared to other methods mentioned in the literature. These results demonstrate that our proposed method is better suited for small retinal datasets, exhibiting improved segmentation accuracy and generalization compared to existing approaches.

A novel Dual-Branch Asymmetric Encoder–Decoder Segmentation Network for accurate colonic crypt segmentation

Colorectal cancer (CRC) is a leading cause of cancer-related deaths, with colonic crypts (CC) being crucial in its development. Accurate segmentation of CC is essential for decisions CRC and developing diagnostic strategies. However, colonic crypts’ blurred boundaries and morphological diversity bring substantial challenges for automatic segmentation. To mitigate this problem, we proposed the Dual-Branch Asymmetric Encoder–Decoder Segmentation Network (DAUNet), a novel and efficient model tailored for confocal laser endomicroscopy (CLE) CC images. In DAUNet, we crafted a dual-branch feature extraction module (DFEM), employing Focus operations and dense depth-wise separable convolution (DDSC) to extract multiscale features, boosting semantic understanding and coping with the morphological diversity of CC. We also introduced the feature fusion guided module (FFGM) to adaptively combine features from both branches using cross-group spatial and channel attention to improve the model representation in focusing on specific lesion features. These modules are seamlessly integrated into the encoder for effective multiscale information extraction and fusion, and DDSC is further introduced in the decoder to provide rich representations for precise segmentation. Moreover, the local multi-layer perceptron (LMLP) module is designed to decouple and recalibrate features through a local linear transformation that filters out the noise and refines features to provide edge-enriched representation. Experimental evaluations on two datasets demonstrate that the proposed method achieves Intersection over Union (IoU) scores of 81.54% and 84.83%, respectively, which are on par with state-of-the-art methods, exhibiting its effectiveness for CC segmentation. The proposed method holds great potential in assisting physicians with precise lesion localization and region analysis, thereby improving the diagnostic accuracy of CRC.

Thermo-metallo-mechanical based phase transformation modeling for high-speed milling of Ti–6Al–4V through stress-strain and temperature effects

The optimization of machining parameters, tool longevity, and surface quality in High-Speed Milling (HSM) of Ti–6Al–4V relies immensely on understanding the local phase transformation. This study endeavors to build a Finite Element (FE) model capable of forecasting phase alterations during the rapid thermal fluctuations intrinsic to Ti–6Al–4V machining. Dynamic phase transformation models were initially introduced to capture rapid heating and cooling phenomena. Using a user-defined subroutine, the phase transitions predictive models were integrated into the HSM simulation within Abaqus/Explicit. Simulation outcomes unveiled phase transitions primarily occurring within the serrated chip and at the tool-workpiece interface. Notably, during rapid heating, when the cutting speed increased to 350 m/min, the β-phase volume fraction surged from 7.5 to 96.38%. A similar trend was observed with feed rate adjustments (i.e., 0.15–0.25 mm/tooth), where β-phase increased from 7.5 to 67.84%. Rapid cooling facilitated the reversion of the transformed β-phase back into the α'-phase. Finally, some advanced characterization techniques were employed to validate the developed thermo-metallo-mechanical coupled FE model for phase transformation. The simulation results verified by the experimental data promotes a better understanding of phase alteration mechanisms and microstructural evolution in HSM of Ti–6Al–4V. The current research is also beneficial for crucial insights into optimizing the machining conditions and their impact on tool-material interactions and surface integrity.

Multiple slips on Darcy–Forchheimer unsteady flow manifested with Cattaneo–Christov heat flux over a stretching sheet

This primary goal of this analysis is to investigate how several slips affect the flow, heat, and mass performance of Darcy–Forchheimer unstable behavior beyond a stretching sheet using a numerical analysis of the Soret effect. The novel constitutive model Cattaneo–Christov is illustrated to analyze the features of thermal relaxation time. The Cattaneo–Christov method is used to predict heat and mass transport. A surface that slanders and has varying thicknesses propels the flow. Through the use of local similarity transformations, the governing nonlinear partial differential equations reduce to the coupled ordinary differential equations and include the momentum, energy, and concentration equations. The altered ODEs are calculated numerically using the Runge–Kutta Fehlberg scheme and an efficient shooting procedure. The physical properties of the temperature, concentration, and fluid velocity profiles are shown visually and shed light on the change of several governing parameters. For example, in comparison to the classical Fourier’s heat model, our result suggests that the Cattaneo–Christov heat flux constitution has lower temperature and thermal boundary layer thickness. In the meantime, a high wall thickness parameter greatly upgrades the rate of heat transfer, and thermal relaxation has the opposite effect. Discussion is held regarding the effects of various miscellaneous variables on temperature, concentration, and velocity.

Evolutionary dynamics of island shoreline in the context of climate change: insights from extensive empirical evidence

ABSTRACT The evolution and future trajectory of island shorelines, amidst global climate change, are of increasing concern to governments, communities and researchers worldwide. However, the field of island studies is often hampered by a lack of data and inconsistent methodologies, leading to an inadequate understanding of the processes driving shoreline changes on islands within the context of climate change. This research aims to bridge this gap by analyzing islands in Southeast Asia, the Indian Ocean and the Mediterranean Sea from 1990 to 2020 using remote sensing. Of over 13,000 islands examined, approximately 12% experienced significant shifts in shoreline positions. The total shoreline length of these islands approaches 200,000 km, with 7.57% showing signs of landward erosion and 6.05% expanding seaward. Human activities, particularly reclamation and land filling, were identified as primary drivers of local shoreline transformations, while natural factors have a comparatively minor impact. Moreover, the ongoing rise in sea levels is identified as an exacerbating factor for coastal erosion rather than the primary cause. Drawing from these findings, we propose several adaptive measures for islands in response to climate change. Taken together, this research provides comprehensive data and a basis for decision-making for sustainable development of island territories.

GoLDFormer: A global–local deformable window transformer for efficient image restoration

Thanks to the powerful modeling capabilities of multi-head self attention (MSA), transformers have shown significant performance gains in vision tasks. However, as transformers require heavy computation, more efficient designs are required. In this paper, we present an efficient transformer architecture named GoLDFormer for image restoration. GoLDFormer extends the capability of window-based self-attention through two core designs. First, We propose a globally-enhanced window-based transformer block (G-WTB), which applies transposed attention to a compressed window representation rather than the spatial features, thus establishing connections between all windows with less computational complexity. Second, since the interactions between image content and window attention weights can be interpreted as spatially varying convolution, we introduce an adaptive filter structure into transformer models and propose a deformable filtering block (DFB) to enable cross-window connections. By adjusting the shape of the generated filters in the DFB, we can balance the computational costs and the degree of adjacent window interaction. Extensive experiments on several image restoration tasks demonstrate that GoLDFormer achieves competitive results against recent methods but with optimal computational costs.

Global–local feature learning for fine-grained food classification based on Swin Transformer

Separable object parts, such as the head and tail in a bird, are vital for fine-grained visual classifications. For those objects without separable parts, the classification task relies only on local and global textural image features. Although the Swin Transformer architecture was proposed to efficiently capture both local and global visual features, it still exhibits a bias towards global features. Therefore, our goal is to enhance the local feature learning capability of the Swin Transformer by adding four new modules of the Local Feature Extraction Network (L-FEN), Convolution Patch-Merging (CP), Multi-Path (MP), and Multi-View (MV). The L-FEN enhances the Swin transformer with the improved local feature capture. The CP is a localized and hierarchical adaptation of the Swin’s Patch Merging technique. The MP method integrates features across various Swin stages to accentuate local details. Meanwhile, the MV Swin transformer block supersedes traditional Swin blocks with those incorporating varied receptive fields, ensuring a broader scope of local feature capture. Our enhanced architecture, named Global–Local Swin Transformer (GL-Swin), is applied to solve a fine-grained food classification task. On three major food datasets: ISIA Food-500 UEC Food-256, and Food-101, our GL-Swin achieved accuracies of 66.75%, 85.78%, and 92.93% respectively, consistently outperforming other leading methods.

DiffGAN: An adversarial diffusion model with local transformer for MRI reconstruction

Magnetic resonance imaging (MRI) is non-invasive and crucial for clinical diagnosis, but it has long acquisition time and aliasing artifacts. Accelerated imaging techniques can effectively reduce the scanning time of MRI, thereby decreasing the anxiety and discomfort of patients. Vision Transformer (ViT) based methods have greatly improved MRI image reconstruction, but their computational complexity and memory requirements for the self-attention mechanism grow quadratically with image resolution, which limits their use for high resolution images. In addition, the current generative adversarial networks in MRI reconstruction are difficult to train stably. To address these problems, we propose a Local Vision Transformer (LVT) based adversarial Diffusion model (Diff-GAN) for accelerating MRI reconstruction. We employ a generative adversarial network (GAN) as the reverse diffusion model to enable large diffusion steps. In the forward diffusion module, we use a diffusion process to generate Gaussian mixture distribution noise, which mitigates the gradient vanishing issue in GAN training. This network leverages the LVT module with the local self-attention, which can capture high-quality local features and detailed information. We evaluate our method on four datasets: IXI, MICCAI 2013, MRNet and FastMRI, and demonstrate that Diff-GAN can outperform several state-of-the-art GAN-based methods for MRI reconstruction.

Deciphering the Local Environment of Electrocatalytic Metal Sites with X‐ray Absorption Fine Structure

Electrocatalysis plays a pivotal role in energy conversion and holds significant promise for the development of new energy sources. Understanding the intricate atomic‐level interplay between active sites and electrocatalytic activity is essential for comprehending catalytic behavior and advancing high‐performance catalysts. In this review, an overview of the recent advances in X‐ray absorption fine structure (XAFS) is provided for deciphering the local environment of electrocatalytic metal sites. The application of XAFS in disclosing the electronic interaction and coordination environment of metal sites is summarized, offering insights into the correlation between ligand arrangement and catalytic activity. Special attention is given to advanced XAFS techniques for exploring active species, including determining the actual active sites, monitoring the local structure transformation, and deciphering metal sites–catalysis relationship. The limitations of traditional XAFS have naturally prompted the development of high‐spatiotemporal‐resolution and high‐energy‐resolution techniques as well as in‐situ/operando methods to better understand the entire catalytic process. This review is anticipated to provide a comprehensive understanding of the capabilities of XAFS in exploring the fine structure of electrocatalysts, with implications for the design of advanced catalytic materials.

Effects of annealing temperature on the microstructure, mechanical and tribological properties of TiAlSiCN coatings

The effects of annealing temperature on the microstructure, mechanical properties, and tribological properties of TiAlSiCN coatings with C contents of 9.24 at.% and 19.45 at.% are systematically investigated. The results show that all coatings maintain cubic structures with strongly preferential orientation of (200) planes, and the microstructure transforms from amorphous embedded in (Ti, Al)(C, N) nanocrystalline to amorphous encapsulated nanocrystalline with increasing C content. Elevating annealing temperature to 800 °C leads to a continuous decline in hardness, whereas the elastic strain resistance H/E increases first and then decreases. Release of H2 formed by the breaking of the C–H bonds at 400 °C induces a local stress transformation from compressive to tensile. The toughness and hardness decreases dramatically at 600 °C by conversion of sp3 to sp2. The wear mechanisms of the coating against Si3N4 counterballs are adhesive and oxidative wear. The coefficient of friction (COF) and wear rate initially decrease followed by an increase. The coatings with a-C structure exhibit lower COF and the wear process is smoother, but showing higher wear rate. Thus, the annealed TiAlSiCN coatings with preferred C content could improve the mechanical and tribological properties.