Mapping Module Research Articles

Correlative Raman-Voltage Microscopy Revealing the Localized Structure-Stress Relationship in Silicon Solar Cells.

Knowledge of localized strain at the micrometer scale is essential for tailoring the electrical and mechanical properties of ongoing thinning of crystal silicon (c-Si) solar cells. Thinning c-Si wafers below 110 μm are susceptible to cracking in manufacturing due to the nonuniform stress distribution at a micrometer region, necessitating a rigorous technique to reveal the localized stress distribution correlating with its device electrical output. In this context, a Raman microscopy integrated with a photovoltage mapping setup with high resolution to the submicrometer scale is developed to acquire correlative Raman-voltage of the localized physical properties at the microcracks on the rear side of c-Si solar cells. By integrating photoelectrical, mechanical, and theoretical simulations, we elucidated the evolution of the microcracks. The localized stresses cause significant electrical output degradation in c-Si solar cells. In addition, theoretical simulations and experimental characterization indicate that the etched rear side acts as a more intense stress concentrator, resulting in an asymmetrical stress distribution between the rear and front sides of c-Si solar cells. This finding provides valuable insights into the origin of microcracks in c-Si solar cells and serves as a metrology tool for microscale mapping of strain-engineered photovoltaic modules.

A hybrid encryption framework leveraging quantum and classical cryptography for secure transmission of medical images in IoT-based telemedicine networks

In the era of the Internet of Things (IoT), the transmission of medical reports in the form of scan images for collaborative diagnosis is vital for any telemedicine network. In this context, ensuring secure transmission and communication is necessary to protect medical data to maintain privacy. To address such privacy concerns and secure medical images against cyberattacks, this research presents a robust hybrid encryption framework that integrates quantum, and classical cryptographic methods. The proposed framework not only secure medical data against cyber threats but also protects the secret security keys. Initially, a Quantum Key Distribution (QKD) is employed to generate a shared key, which is then used to secure the symmetric keys via One-Time Pad (OTP) encryption. Next, bit-planes are extracted from each color component. The rows and columns of the extracted bit-planes are scrambled using random sequences which are generated by a 6D hyperchaotic Chen system and the Ikeda map. To further increase confusion in the original data, multiple-step pixel scrambling operations such as pixel shuffling, pixel value shuffling, and rotational and flipping operations are implemented. After the confusion phase, a combination of affine transformations with non-linear functions, Discrete Cosine Transform (DCT) with complex modulation, Discrete Wavelet Transform (DWT) with random phase modulation, bilinear transformation, and nonlinear polynomial mapping are employed to create diffusion in the scrambled components. These multiple encryption operations aim to maximize randomness in the final ciphertext image. Additionally, to reduce computational complexity, only the Most Significant Bit-Planes (MSBs) are encrypted, as they contain more than 94% of the plaintext information. Several experimental results and analyses are conducted to assess the proposed encryption framework, including entropy analysis, key sensitivity analysis, correlation analysis lossless analysis, and histogram analysis. Furthermore, the framework is tested against various cyberattacks such as brute-force attacks, clipping attacks, and noise attacks on the ciphertext images, to demonstrate its resilience against such threats.

A Secure Data Hiding For H.264 Video Based on Chaotic Map Methods and RC4 Algorithm

Video steganography is the common field of data security in digital communication where data include text, image, and video. Compressed video is the suitable modern source for hiding huge secret data that will trust security. This work proposed a compressed video steganography scheme for hiding cipher secret data in high security levels. The proposed work consists of three phases, protected secret data (Text English and Arabic, Images, and Video) based on proposed chaotic maps module and RC4 algorithm. The chaotic maps module is a combination between (honen map and sine map) with simple operations to protected plain data. Second phase, hidden cipher secret data between the quantized discrete cosine transform (QDCT) and entropy coding from the inter-prediction mode at the H.264/AVC video frames using Least Significant Bit (LSB) replacement. Third phase, transmit and receive secret data. The experimental result shows that the proposed scheme is high-security level where was cipher secret data with the average entropy value of (7.993), an average PSNR value of (8.790 dB), an average correlation value of (3.4x 10-3), key length is 228 bit and key space is 2228. Moreover, measured Randomness to generation key by NIST were succeeded in all the tests and Robustness, imperceptibility, and embedding capacity are high while maintaining visual quality, where the average PSNR value of (36.26 dB), and the average SSIM value of (0.95).

The Spectrum Difference Enhanced Network for Hyperspectral Anomaly Detection

Most deep learning-based hyperspectral anomaly detection (HAD) methods focus on modeling or reconstructing the hyperspectral background to obtain residual maps from the original hyperspectral images. However, these methods typically do not pay enough attention to the spectral similarity in the complex environment, resulting in inadequate distinction between background and anomalies. Moreover, some anomalies and background are different objects, but they are sometimes recognized as the objects with the same spectrum. To address the issues mentioned above, this paper proposes a Spectrum Difference Enhanced Network (SDENet) for HAD, which employs variational mapping and Transformer to amplify spectrum differences. The proposed network is based on the encoder–decoder structure, which contains a CSWin-Transformer encoder, Variational Mapping Module (VMModule), and CSWin-Transformer decoder. First, the CSWin-Transformer encoder and decoder are designed to supplement image information by extracting deep and semantic features, where a cross-shaped window self-attention mechanism is designed to provide strong modeling capability with minimal computational cost. Second, in order to enhance the spectral difference characteristics between anomalies and background, a randomly sampling VMModule is presented for feature space transformation. Finally, all fully connected mapping operations are replaced with convolutional layers to reduce the model parameters and computational load. The effectiveness of the proposed SDENet is verified on three datasets, and experimental results show that it achieves better detection accuracy and lower model complexity compared with existing methods.

SpaGAN: A spatially-aware generative adversarial network for building generalization in image maps

Building generalization is an essential task in generating multi-scale topographic maps. The progress of deep learning offers a new paradigm to overcome the coordination challenges faced by conventional building generalization algorithms. Some studies have confirmed the feasibility of several original semantic segmentation networks, such as U-Net and its variants and the conditional generative adversarial network (cGAN), for building generalization in image maps. However, they suffer from critical deformation effects, especially for large and geometrically complex buildings. Since learning building generalization essentially means modeling the subtle transformation of building footprints across scales, we argue that the spatial awareness of a neural network, for instance, regarding building size and shape, is crucial to effective learning. Thus, we propose a spatially-aware generative adversarial network, SpaGAN. It takes a representative cGAN, pix2pix, as the backbone, and modifies two modules: In the U-Net-based generator, an atrous spatial pyramid pooling (ASPP) module replaces the conventional convolutional module to extract multi-scale features of buildings of varying sizes and shapes; in the PatchGAN-based discriminator, a signed distance map (SDM) module is used to capture the fine-grained shape difference for discrimination. The proposed network was comprehensively evaluated with a synthetic and a real-world dataset. The results demonstrate that SpaGAN outperforms existing baseline models (U-Net, ResU-Net, pix2pix) for building generalization, particularly in the real-world dataset. The new model can achieve more reasonable aggregation, simplification, and squaring generalization operators.

DeCGR: an interactive toolkit for deciphering complex genomic rearrangements from Hi-C data

BackgroundComplex genomic rearrangements (CGRs) drive the restructuring of chromatin architecture, resulting in significant interactions among rearranged fragments, visible as anomalous interaction blocks in chromatin contact maps generated by chromosome conformation capture technologies such as Hi-C. These blocks not only offer the orientation and genome coordinates of rearranged fragments but also filter out false positive CGRs, thereby facilitating CGR assembly. Despite this, there is a lack of interactive graphical software tailored for this purpose.ResultsWe present DeCGR, a user-friendly Python toolbox specifically designed for deciphering CGRs in Hi-C data. DeCGR consists of four independent execution components. The Breakpoint Filtering module identifies and filters simple rearrangements, providing the coordinates of rearrangement breakpoints. The Fragment Assembly module automatically assembles CGRs and visualizes the assembly process, facilitating the direct association between anomalous interaction blocks and CGR events. The Validation CGRs module verifies the completeness and accuracy of CGRs by generating the Hi-C map with CGRs through a simulation process and examines the difference from the original Hi-C maps. This module displays both the original and the simulated Hi-C map with highlighted rearranged fragment boundaries for rapid review to assess the CGRs. Finally, the Reconstruct Hi-C Map module provides the reconstructed Hi-C map based on the determined CGRs, allowing users to directly observe the impact of rearrangements on chromatin structure.ConclusionsDeCGR is designed specifically for biologists who aim to explore CGRs from Hi-C data. It provides a validation module to ensure the completeness and correctness of CGRs. Additionally, it allows users to generate CGR assembly results and reconstruct the Hi-C map with just one click. DeCGR provides intuitive visualization results for each module, allowing users to easily associate CGRs with Hi-C maps. DeCGR is operable through a user-friendly graphical interface. Source codes are freely available at https://github.com/GaoLabXDU/DeCGR.

PDCA-Net: Parallel dual-channel attention network for polyp segmentation

Accurate segmentation of polyps in colonoscopy images is crucial for the diagnosis and cure of colorectal cancer. Although various deep learning methods have been proposed and have shown promising performance, accurately distinguishing between polyp and mucosal boundaries remains a challenge. In this work, we propose a Parallel Dual-Channel Attention Network (PDCA-Net) for polyp segmentation. This method utilizes the mapping transformations to adaptively encapsulate the global dependency from superpixel into pixels, enhancing the model’s ability to localize foreground and background regions. Specifically, we first design a parallel spatial and channel attention fusion module to capture the global dependencies at the superpixel level from the spatial and channel dimensions. Furthermore, an adaptive associative mapping module is proposed to encapsulate the global dependencies of superpixels into each pixel through a coarse-to-fine learning strategy. Extensive experiments demonstrate that the proposed PDCA-Net effectively improves the segmentation performance and achieves new state-of-the-art results (i.e., 0.815, 0.936, 0.945, and 0.838 mDice, 0.744, 0.891, 0.900, and 0.765 mIoU on the ETIS, Kvasir-SEG, CVC-ClinicDB, and CVC-ColonDB). Our code is available at https://github.com/lzucg/PDCA-Net.

Hierarchical Scale Awareness for object detection in Unmanned Aerial Vehicle Scenes

Existing object detection models are typically designed without considering the small-scale context, leading to significant challenges in detecting small objects within Unmanned Aerial Vehicle (UAV) scenes. Therefore, this paper aims to incorporate a novel hierarchical scale-aware module into the neck component of the classical YOLO architecture. This module hierarchically enhances the object features, progressing from small to large scales. Specifically, the proposed Small-Scale Awareness (SSA) module is designed to enhance features from small-scale objects, while the introduced Receptive Field Expansion (RFE) module is responsible for modeling contextual information in a way that expands the receptive field while maintaining feature diversity for large-scale objects. Additionally, in the backbone of our model, a Stack of Non-Linear Mapping (SNM) module is proposed, which utilizes deformable convolutions to fuse feature maps of diverse scales through a cascade of non-linear mapping units, to capture a wide range of contextual and discriminative information. The experimental results on the VisDrone dataset demonstrate that the proposed model outperforms the state-of-the-art models both on the mean Average Precision (mAP) and Average Precision 50 (AP50) metrics. The ablation studies have proved that the proposed modules are beneficial to improve the detection performance of objects in UAV scenes.

Prompt-Based Modality Bridging for Unified Text-to-Face Generation and Manipulation

Text-driven face image generation and manipulation are significant tasks. However, such tasks are quite challenging due to the gap between text and image modalities. It is difficult to utilize current methods to deal with both of the two problems because these methods are usually designed for one certain task, limiting their application in real scenarios. To address the two problems in one framework, we propose a Unified Prompt-based Cross-Modal Framework (UPCM-Frame) to bridge the gap between the text modality and image modality with CLIP and StyleGAN, which are two large-scale pre-trained models. The proposed framework is combined with two main modules: a Text Embedding-to-Image Embedding projection module based on a special prompt embedding pair, and a projection module mapping Image Embeddings to semantically aligned StyleGAN Embeddings which can be used in both image generation and manipulation. The proposed framework is able to handle complicated descriptions and generate impressive results with high quality due to the utilization of large-scale pre-trained models. In order to demonstrate the effectiveness of the proposed method in the two tasks, we conduct experiments to evaluate the results of our method both quantitatively and qualitatively.

Multi-module echo state network with variable skip length for chaotic time series prediction

Echo state networks (ESNs) have been extensively applied in time series prediction problems. However, the memory-nonlinearity trade-off problem severely limits the ability of ESNs to deal with chaotic time series prediction problems. In this study, a multi-module echo state network with variable skip length (MESN-VSL) is proposed to address this problem. First, the reservoir is divided into a nonlinear mapping module and multiple linear memory modules based on the idea of memory and nonlinearity separation. This idea can effectively balance the memory-nonlinearity trade-off problems. Second, a multi-module mechanism with skip length is put forward to model the characteristics of chaotic time series. The skip length and the number of linear memory modules of the MESN-VSL model are automatically determined based on the idea of phase-space reconstruction. Finally, the experimental results further demonstrate that the MESN-VSL model is superior to some existing models in chaotic time series prediction.

Semantic Guided Matting Net

Abstract Human matting refers to extracting human parts from natural images with high quality, including human detail information such as hair, glasses, hats, etc. This technology plays an essential role in image synthesis and visual effects in the film industry. When the green screen is not available, the existing human matting methods need the help of additional inputs (such as trimap, background image, etc.), or the model with high computational cost and complex network structure, which brings great difficulties to the application of human matting in practice. To alleviate such problems, we use a segmentation network as the foundation and use multiple branches to achieve human segmentation, contour detail extraction, and information fusion. We also propose a foreground probability map module, which uses the feature maps in the segmentation network to pre-estimate the foreground probabilities of each pixel and obtain Semantic Guided Matting Net. Under the condition that only a single image is needed as the input, the human matting task can be realized by making full use of the semantic information in the image. We validate our method on the P3M-10k dataset. Compared with the benchmark, our method has made significant improvements in various evaluation indicators.

Accuracy of endocardial abnormal electrogram characterization using a 17-AHA-segmental late boundary and voltage-dependent sensitivity annotation for ventricular tachycardia ablation

Abstract Background Substrate-based catheter ablation of ventricular tachycardia (VT) has become the mainstay therapy for patients with scar-related VT. Correct identification and timing of abnormal near-field EGMs and discerning it from far-field EGMs, is key for procedural success. The novel CARTO™ Late Activation Mapping (LAM) module provides fast and accurate annotation of near-field EGMs including late abnormal ventricular activation. For the algorithm to succeed, a late boundary (LB) annotation line should be positioned immediately behind the normal near-field EGM, however, since the timing of normal local near-field activation varies greatly throughout the endocardium, there is currently no consensus for the optimal LB positioning. We hypothesized that 17-AHA segmental annotation of the endocardial LB followed by scar-dependent sensitivity settings, will improve the accuracy of the near-field EGM characterization. Purpose To evaluate the accuracy of endocardial abnormal EGM characterization using a 17-AHA-segmental late boundary and voltage-dependent sensitivity annotation as compared to conventional mid-QRS LB positioning. Methods Five high-resolution substrate voltage maps were obtained from 5 patients (80% male, 68±7 years old, 80% ischemic cardiomyopathy) that underwent a catheter-based ablation for scar-related VT. A personalized CT-derived anatomical reconstruction of the 17-segment AHA model was co-registered offline with the electroanatomic voltage map, using ADAS3D software. Per segment, individual LB timings were set at the highest sensitivity. A gold standard annotation map was created by two independent experts after manual scrutiny of individual EGMs. Next, the expert map was compared to automatic maps created from segmental LB and mid-QRS LB positions at normal, high, highest or mixed (normal/high or normal/highest) sensitivities, focusing on EGM voltage and activation accuracy. Results A mean of 2041±1028 points per map were analyzed. Correct correlation of EGM annotation was found on average in 91% (activation) and 81% (voltage) for all segmental LB positioned maps (segLB), and in 83% (activation) and 77% (voltage) for all mid-QRS LB positioned maps (mid-QRSLB). The highest correlation with expert map for both activation and voltage annotation was found in the mixed sensitivity normal/highest map with segmental LB positioning (activation 98%, voltage 88%). The correlation on activation was significantly superior in the segLB map when compared to the normal sensitivity maps (p=0.0008 for mid-QRSLB and p=0.02 for segLB). For correlation on voltage, the segLBmap with mixed sensitivity was significantly superior compared to three mid-QRSLB maps at normal (p=0.04), high and highest (p=0.01) sensitivity. Conclusion Individualized, 17-AHA-segmental late boundary and voltage-dependent sensitivity annotation provides a possible accurate and clinically-relevant framework for substrate assessment. Prospective validation is needed.

Unsupervised generative model for simulating post-operative double eyelid image.

Simulating the outcome of double eyelid surgery is a challenging task. Many existing approaches rely on complex and time-consuming 3D digital models to reconstruct facial features for simulating facial plastic surgery outcomes. Some recent research performed a simple affine transformation approach based on 2D images to simulate double eyelid surgery outcomes. However, these methods have faced challenges, such as generating unnatural simulation outcomes and requiring manual removal of masks from images. To address these issues, we have pioneered the use of an unsupervised generative model to generate post-operative double eyelid images. Firstly, we created a dataset involving pre- and post-operative 2D images of double eyelid surgery. Secondly, we proposed a novel attention-class activation map module, which was embedded in a generative adversarial model to facilitate translating a single eyelid image to a double eyelid image. This innovative module enables the generator to selectively focus on the eyelid region that differentiates between the source and target domain, while enhancing the discriminator's ability to discern differences between real and generated images. Finally, we have adjusted the adversarial consistency loss to guide the generator in preserving essential features from the source image and eliminating any masks when generating the double eyelid image. Experimental results have demonstrated the superiority of our approach over existing state-of-the-art techniques.

Hyperspectral Image Classification Algorithm for Forest Analysis Based on a Group-Sensitive Selective Perceptual Transformer

Substantial advancements have been achieved in hyperspectral image (HSI) classification through contemporary deep learning techniques. Nevertheless, the incorporation of an excessive number of irrelevant tokens in large-scale remote sensing data results in inefficient long-range modeling. To overcome this hurdle, this study introduces the Group-Sensitive Selective Perception Transformer (GSAT) framework, which builds upon the Vision Transformer (ViT) to enhance HSI classification outcomes. The innovation of the GSAT architecture is primarily evident in several key aspects. Firstly, the GSAT incorporates a Group-Sensitive Pixel Group Mapping (PGM) module, which organizes pixels into distinct groups. This allows the global self-attention mechanism to function within these groupings, effectively capturing local interdependencies within spectral channels. This grouping tactic not only boosts the model’s spatial awareness but also lessens computational complexity, enhancing overall efficiency. Secondly, the GSAT addresses the detrimental effects of superfluous tokens on model efficacy by introducing the Sensitivity Selection Framework (SSF) module. This module selectively identifies the most pertinent tokens for classification purposes, thereby minimizing distractions from extraneous information and bolstering the model’s representational strength. Furthermore, the SSF refines local representation through multi-scale feature selection, enabling the model to more effectively encapsulate feature data across various scales. Additionally, the GSAT architecture adeptly represents both global and local features of HSI data by merging global self-attention with local feature extraction. This integration strategy not only elevates classification precision but also enhances the model’s versatility in navigating complex scenes, particularly in urban mapping scenarios where it significantly outclasses previous deep learning methods. The advent of the GSAT architecture not only rectifies the inefficiencies of traditional deep learning approaches in processing extensive remote sensing imagery but also markededly enhances the performance of HSI classification tasks through the deployment of group-sensitive and selective perception mechanisms. It presents a novel viewpoint within the domain of hyperspectral image classification and is poised to propel further advancements in the field. Empirical testing on six standard HSI datasets confirms the superior performance of the proposed GSAT method in HSI classification, especially within urban mapping contexts, where it exceeds the capabilities of prior deep learning techniques. In essence, the GSAT architecture markedly refines HSI classification by pioneering group-sensitive pixel group mapping and selective perception mechanisms, heralding a significant breakthrough in hyperspectral image processing.

A Training-Free Latent Diffusion Style Transfer Method

Diffusion models have attracted considerable scholarly interest for their outstanding performance in generative tasks. However, current style transfer techniques based on diffusion models still rely on fine-tuning during the inference phase to optimize the generated results. This approach is not merely laborious and resource-demanding but also fails to fully harness the creative potential of expansive diffusion models. To overcome this limitation, this paper introduces an innovative solution that utilizes a pretrained diffusion model, thereby obviating the necessity for additional training steps. The scheme proposes a Feature Normalization Mapping Module with Cross-Attention Mechanism (INN-FMM) based on the dual-path diffusion model. This module employs soft attention to extract style features and integrate them with content features. Additionally, a parameter-free Similarity Attention Mechanism (SimAM) is employed within the image feature space to facilitate the transfer of style image textures and colors, while simultaneously minimizing the loss of structural content information. The fusion of these dual attention mechanisms enables us to achieve style transfer in texture and color without sacrificing content integrity. The experimental results indicate that our approach exceeds existing methods in several evaluation metrics.

SELFNet: Denoising Shear Wave Elastography Using Spatial-temporal Fourier Feature Networks

ObjectiveUltrasound-based shear wave elastography offers estimation of tissue stiffness through analysis of the propagation of a shear wave induced by a stimulus. Displacement or velocity fields during the process can contain noise as a result of the limited number of acquisitions. With advances in physics-informed deep learning, neural networks can approximate a physics field by minimizing the residuals of governing physics equations. MethodsIn this research, we introduce a shear wave elastography Fourier feature network (SELFNet) using spatial-temporal random Fourier features within a physics-informed neural network framework to estimate and denoise particle displacement signals. The network uses a sparse mapping to increase robustness and incorporates the governing equations for regularization while simultaneously learning the mapping of the shear modulus. The method was evaluated in datasets from tissue-mimicking phantom of lesions and ex vivo tissue. ResultsThe findings indicate that SELFNet is capable of smoothing out the noise in phantom lesions with different stiffness and sizes, outperforming a reference Gaussian filtering method by 17% in relative ℓ2 error, 45% in reconstruction root-mean-square error. Furthermore, the ablation study suggested that SELFNet can prevent over-fitting through the Fourier feature mapping module. An ex vivo study confirmed its applicability to different types of tissue. ConclusionThe implementation of SELFNet shows promise for shear wave elastography with limited acquisitions. In this context, subject to successful translation, it has the potential to be extended to clinical applications, such as the diagnosis of cancer or liver disease.

Mapping and Localization of Autonomous Mobile Robots in Simulated Indoor Environments

Autonomously making a map, localizing within it, and planning with it are fundamental problems in mobile robotics. Every autonomous mobile robot system must include a solution to all three problems. These three problems are interconnected, with simultaneous localization and mapping (SLAM) being a well-known issue. However, there is indeed a growing and developing realization in the research field that path planning how a robot goes about mapping and finding an environment (and then operating in the environment such as starting to the destination point) can avoid degenerate conditions and greatly reduce SLAM complexity. In this paper, the implementation of an autonomous mobile robot system for indoor environments using open-source ROS packages and a combination of cartography algorithm and adaptive Monte Carlo localization (AMCL) algorithms has been implemented. The system addresses the challenge of developing three components such as mapping, localization, and path planning systems for indoor autonomous mobile robots. The mapping module creates a global map using the cartography ROS package and SLAM algorithm. The localization module estimates the robot's pose using the AMCL approach. The planning module generates collision-free trajectories and control commands using the moving base ROS package. The experimental results demonstrate the effectiveness of this approach and its valuable contribution to the robotics field. The cartography algorithm mapping algorithm generates accurate and reliable maps, while the localization algorithm successfully determines the robot's position with good performance. Additionally, the path planning algorithm effectively avoids both static and dynamic obstacles, ensuring smooth navigation in the environment.

DMMGNet: A discrimination mapping and memory bank mean guidance-based network for high-performance few-shot industrial anomaly detection

For deep learning-based industrial anomaly detection, it is still challenging to get adequate images for model training and achieve cold start for cross-product migration, restricting their practical application in real industrial production. Herein, an innovative few-shot anomaly detection network DMMGNet based on discrimination mapping and memory bank mean guidance strategies are demonstrated, which is trained by a new two-branch data augmentation technique. By separating the features stored in memory bank from the features used for training, the two-branch data augmentation method can significantly improve the robustness of few-shot model training and reduce the redundance of memory bank. In the elaborately designed discrimination mapping module, new negative samples are generated by adding dynamic Gaussian noise to normal samples along the channel dimension in feature space to solve the problem of sample imbalance. Meanwhile, the discrimination mapping module also helps to map the feature distribution of positive samples to the target domain more efficiently and reduce the deviation of feature domain, conducive to a more precise separation of positive and negative samples. In addition, a novel mean guidance approach with an optimized loss function is developed to guide the positive sample feature mapping by specifying the local feature space center to form a clear feature domain contour and enhance the detection accuracy. The multiple experimental results validate that our DMMGNet outperforms the most advanced anomaly detection counterparts on image-level AUROC, showing an increase by 0.3–3 % on both MVTec AD and MPDD benchmarks under several few-shot scenarios.

Weakly supervised semantic segmentation by knowledge graph inference

The weakly supervised semantic segmentation (WSSS) training based on image-level labels in convolutional neural network (CNN) is usually divided into two stages: multi-label classification and semantic segmentation. However, most of the existing work focuses on the improvement of the multi-label classification network stage, and little effort has been done to improve the performance of the downstream segmentation networks. In addition, CNN-based local convolution lacks in modeling extensive dependencies among categories. Therefore, in this paper, we propose a graph reasoning method to improve both the upstream and the downstream stages of the multi-label classification network and the semantic segmentation networks. In the multi-label classification network, we utilize external knowledge combined with a graph convolutional network (GCN) to perform global reasoning on the dependencies of each category. In the segmentation network, the Graph Reasoning Mapping Module (GRM) is proposed to explore the knowledge acquired from text corpora and facilitate contextual reasoning in various categories of image regions. The proposed GRM module is able to enhance the feature representation of local convolutions on the high-level semantics of the segmentation network, and adaptively learn the semantic consistency of each sample. We achieve state-of-the-art WSSS performance on PASCAL VOC 2012 and MS-COCO 2014 datasets with only image-level supervision. Extensive experiments on multi-label classification networks and semantic segmentation networks demonstrate the effectiveness of our proposed graph reasoning method on WSSS. Our code is available at:https://github.com/JIA-ZHANG666/GRM_layer.

G[formula omitted]BFNN: Generalized geodesic basis function neural network

Real-world data is typically distributed on low-dimensional manifolds embedded in high-dimensional Euclidean spaces. Accurately extracting spatial distribution features on general manifolds that reflect the intrinsic characteristics of data is crucial for effective feature representation. Therefore, we propose a generalized geodesic basis function neural network (G2BFNN) architecture. The generalized geodesic basis functions (G2BF) are defined based on generalized geodesic distances. The generalized geodesic distance metric (G2DM) is obtained by learning the manifold structure. To implement this architecture, a specific G2BFNN, named discriminative local preserving projection-based G2BFNN (DLPP-G2BFNN) is proposed. DLPP-G2BFNN mainly contains two modules, namely the manifold structure learning module (MSLM) and the network mapping module (NMM). In the MSLM module, a supervised adjacency graph matrix is constructed to constrain the learning of the manifold structure. This enables the learned features in the embedding subspace to maintain the manifold structure while enhancing the discriminability. The features and G2DM learned in the MSLM are fed into the NMM. Through the G2BF in the NMM, the spatial distribution features on manifold are obtained. Finally, the output of the network is obtained through the fully connected layer. Compared with the local response neural network based on Euclidean distance, the proposed network can reveal more essential spatial structure characteristics of the data. Meanwhile, the proposed G2BFNN is a generalized network architecture that can be combined with any manifold learning method, showcasing high scalability. The experimental results demonstrate that the proposed DLPP-G2BFNN outperforms existing methods by utilizing fewer kernels while achieving higher recognition performance.