Guidance Module Research Articles

ViTGaze: gaze following with interaction features in vision transformers

Gaze following aims to interpret human-scene interactions by predicting the person’s focal point of gaze. Prevailing approaches often adopt a two-stage framework, whereby multi-modality information is extracted in the initial stage for gaze target prediction. Consequently, the efficacy of these methods highly depends on the precision of the previous modality extraction. Others use a single-modality approach with complex decoders, increasing network computational load. Inspired by the remarkable success of pre-trained plain vision transformers (ViTs), we introduce a novel single-modality gaze following framework called ViTGaze. In contrast to previous methods, it creates a novel gaze following framework based mainly on powerful encoders (relative decoder parameters less than 1%). Our principal insight is that the inter-token interactions within self-attention can be transferred to interactions between humans and scenes. Leveraging this presumption, we formulate a framework consisting of a 4D interaction encoder and a 2D spatial guidance module to extract human-scene interaction information from self-attention maps. Furthermore, our investigation reveals that ViT with self-supervised pre-training has an enhanced ability to extract correlation information. Many experiments have been conducted to demonstrate the performance of the proposed method. Our method achieves state-of-the-art performance among all single-modality methods (3.4% improvement in the area under curve score, 5.1% improvement in the average precision) and very comparable performance against multi-modality methods with 59% fewer parameters.

Edge-guided multi-scale adaptive feature fusion network for liver tumor segmentation.

Automated segmentation of liver tumors on CT scans is essential for aiding diagnosis and assessing treatment. Computer-aided diagnosis can reduce the costs and errors associated with manual processes and ensure the provision of accurate and reliable clinical assessments. However, liver tumors in CT images vary significantly in size and have fuzzy boundaries, making it difficult for existing methods to achieve accurate segmentation. Therefore, this paper proposes MAEG-Net, a multi-scale adaptive feature fusion liver tumor segmentation network based on edge guidance. Specifically, we design a multi-scale adaptive feature fusion module that effectively incorporates multi-scale information to better guide the segmentation of tumors of different sizes. Additionally, to address the problem of blurred tumor boundaries in images, we introduce an edge-aware guidance module to improve the model's feature learning ability under these conditions. Evaluation results on the liver tumor dataset (LiTS2017) show that our method achieves a Dice coefficient of 71.84% and a VOE of 38.64%, demonstrating the best performance for liver tumor segmentation in CT images.

A Multi-Task Transformer With Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis.

Breast cancer, as a malignant tumor disease, has maintained high incidence and mortality rates over the years. Ultrasonography is one of the primary methods for diagnosing early-stage breast cancer. However, correctly interpreting breast ultrasound images requires massive time from physicians with specialized knowledge and extensive experience. Recently, deep learning-based method have made significant advancements in breast tumor segmentation and classification due to their powerful fitting capabilities. However, most existing methods focus on performing one of these tasks separately, and often failing to effectively leverage information from specific tumor-related areas that hold considerable diagnostic value. In this study, we propose a multi-task network with local-global feature interaction and multiple tumoral region guidance for breast ultrasound-based tumor segmentation and classification. Specifically, we construct a dual-stream encoder, paralleling CNN and Transformer, to facilitate hierarchical interaction and fusion of local and global features. This architecture enables each stream to capitalize on the strengths of the other while preserving its unique characteristics. Moreover, we design a multi-tumoral region guidance module to explicitly learn long-range non-local dependencies within intra-tumoral and peri-tumoral regions from spatial domain, thus providing interpretable cues beneficial for classification. Experimental results on two breast ultrasound datasets show that our network outperforms state-of-the-art methods in tumor segmentation and classification tasks. Compared with the second-best competitive method, our network improves the diagnosis accuracy from 73.64% to 80.21% on a large external validation dataset, which demonstrates its superior generalization capability.

Multi-scale context-aware and boundary-guided image manipulation detection method

Aiming at the problems of traditional image manipulation detection methods, such as fuzzy boundaries, single scale of extracted features, and ignoring background information, this paper proposes an image manipulation detection method based on multi-scale context-aware and boundary-guided. First, spatial details and base features of manipulated images are extracted using an improved pyramid vision transformer. Second, information related to the edge of the falsified region is explored by an edge contextaware module to generate an edge prediction map. Again, the edge guidance module is utilized to highlight the key channels in the extracted features and reduce the interference of redundant channels. Then, the rich contextual information of the manipulated region is learned from multiple sensory fields through the multi-scale context-aware module. Finally, the feature fusion module is utilized to accurately segment the manipulated region by focusing alternately on the foreground and background of the manipulated images. Comparing this paper's method quantitatively and qualitatively on five commonly used public image manipulation detection datasets, the experimental results show that this paper's method can effectively detect manipulated regions and outperforms other methods.

PST-Diff: Achieving High-Consistency Stain Transfer by Diffusion Models With Pathological and Structural Constraints.

Histopathological examinations heavily rely on hematoxylin and eosin (HE) and immunohistochemistry (IHC) staining. IHC staining can offer more accurate diagnostic details but it brings significant financial and time costs. Furthermore, either re-staining HE-stained slides or using adjacent slides for IHC may compromise the accuracy of pathological diagnosis due to information loss. To address these challenges, we develop PST-Diff, a method for generating virtual IHC images from HE images based on diffusion models, which allows pathologists to simultaneously view multiple staining results from the same tissue slide. To maintain the pathological consistency of the stain transfer, we propose the asymmetric attention mechanism (AAM) and latent transfer (LT) module in PST-Diff. Specifically, the AAM can retain more local pathological information of the source domain images, while ensuring the model's flexibility in generating virtual stained images that highly confirm to the target domain. Subsequently, the LT module transfers the implicit representations across different domains, effectively alleviating the bias introduced by direct connection and further enhancing the pathological consistency of PST-Diff. Furthermore, to maintain the structural consistency of the stain transfer, the conditional frequency guidance (CFG) module is proposed to precisely control image generation and preserve structural details according to the frequency recovery process. To conclude, the pathological and structural consistency constraints provide PST-Diff with effectiveness and superior generalization in generating stable and functionally pathological IHC images with the best evaluation score. In general, PST-Diff offers prospective application in clinical virtual staining and pathological image analysis.

Dual phase hierarchical brain tumour detection and segmentation using unet based skip guidance residual convolutional transformer

Brain tumour segmentation and classification is an important approach to understanding tumour characteristics effectively. Accurate lesion segmentation requires many image modalities with varied contrasts. As a result, manual segmentation may not be suitable for large-scale research. The research has shown interest in developing various learning algorithms to detect and segment brain tumour (BT) images accurately based on the set of experimental images. Some of the issues noticed in the existing models, such as high computational complexity, high processing time, and less accuracy, were identified. In order to overcome these existing issues, a novel Dual-phase channel attentional Residual convolutional transformer Unet model (DCARCTUnet) is introduced for segmenting brain tumours. Initially, the input images are passed into the pre-processing stage to remove noise and blur. The Efficient Net model identified the pre-processed image as either normal or tumour. The proposed segmentation model can be integrated into three stages, namely dual phase channel attention module (DCAM), residual convolutional transformer Unet model and Pyramid skip guidance module (PSM). The proposed method uses the BRATS 2020–2021 dataset to analyze the results accurately. The results showed that the proposed model has a dice coefficient value of 0.99 and a Jaccard coefficient value of 0.98. Furthermore, performance analysis is carried out to demonstrate the effectiveness of the proposed model.

Joint-task learning framework with scale adaptive and position guidance modules for improved household rooftop photovoltaic segmentation in remote sensing image

Inaccurate edge detection is a common challenge in the segmentation of household rooftop photovoltaic (PV) systems from remote sensing images, which hinders the accurate retrieval of PV distribution information critical for planning and managing PV development. A widely adopted solution is to incorporate an additional edge detection task into a joint-task learning framework to enhance edge perception. However, existing joint-task learning methods often struggle to accurately detect PV edges and lack effective mechanisms for distinguishing PV edges from those of similar objects. To address the above challenges, we develop a novel joint-task learning framework. This framework introduces a Scale Adaptive Module (SAM) that dynamically adjusts the receptive field of edge features based on the PV actual size and shape, enabling precise detection of PV edges with varying shapes and sizes. In addition, a Position Guidance Module (PGM) is proposed based on the intrinsic relationship between the PV segmentation task and the edge detection task. The PGM not only guides the edge detection task to focus on identifying the semantic edges of PVs using the distribution information from the segmentation task but also enhances the ability of the segmentation task to accurately locate PVs in complex backgrounds by utilizing the backward gradient from the edge detection task. Multiple rounds of repeated experiments on the Duke and IGN datasets demonstrate the framework's superior performance. Compared to other models, the proposed framework significantly improves the detection accuracy of various PV edges, achieving the best performance in household rooftop PV segmentation with an Intersection over Union (IoU) of 77.4 %. This study provides valuable insights into the accurate acquisition of household rooftop PV information and offers a promising solution for object segmentation tasks facing the challenge of inaccurate edge extraction.

DMGNet: Depth mask guiding network for RGB-D salient object detection

Though depth images can provide supplementary spatial structural cues for salient object detection (SOD) task, inappropriate utilization of depth features may introduce noisy or misleading features, which may greatly destroy SOD performance. To address this issue, we propose a depth mask guiding network (DMGNet) for RGB-D SOD. In this network, a depth mask guidance module (DMGM) is designed to pre-segment the salient objects from depth images and then create masks using pre-segmented objects to guide the RGB subnetwork to extract more discriminative features. Furthermore, a feature fusion pyramid module (FFPM) is employed to acquire more informative fused features using multi-branch convolutional channels with varying receptive fields, further enhancing the fusion of cross-modal features. Extensive experiments on nine benchmark datasets demonstrate the effectiveness of the proposed network.

Validity and Reliability of the Choice Theory Reality Therapy Group Guidance Module for School Truancy

Validity and Reliability of the Choice Theory Reality Therapy Group Guidance Module for School Truancy

Semi-Supervised Remote Sensing Building Change Detection with Joint Perturbation and Feature Complementation

The timely updating of the spatial distribution of buildings is essential to understanding a city’s development. Deep learning methods have remarkable benefits in quickly and accurately recognizing these changes. Current semi-supervised change detection (SSCD) methods have effectively reduced the reliance on labeled data. However, these methods primarily focus on utilizing unlabeled data through various training strategies, neglecting the impact of pseudo-changes and learning bias in models. When dealing with limited labeled data, abundant low-quality pseudo-labels generated by poorly performing models can hinder effective performance improvement, leading to the incomplete recognition results of changes to buildings. To address this issue, we propose a feature multi-scale information interaction and complementation semi-supervised method based on consistency regularization (MSFG-SemiCD), which includes a multi-scale feature fusion-guided change detection network (MSFGNet) and a semi-supervised update method. Among them, the network facilitates the generation of multi-scale change features, integrates features, and captures multi-scale change targets through the temporal difference guidance module, the full-scale feature fusion module, and the depth feature guidance fusion module. Moreover, this enables the fusion and complementation of information between features, resulting in more complete change features. The semi-supervised update method employs a weak-to-strong consistency framework to achieve model parameter updates while maintaining perturbation invariance of unlabeled data at both input and encoder output features. Experimental results on the WHU-CD and LEVIR-CD datasets confirm the efficacy of the proposed method. There is a notable improvement in performance at both the 1% and 5% levels. The IOU in the WHU-CD dataset increased by 5.72% and 6.84%, respectively, while in the LEVIR-CD dataset, it improved by 18.44% and 5.52%, respectively.

Experiences of Using a Digital Guidance and Assessment Tool (the Technology-Optimized Practice Process in Nursing Application) During Clinical Practice in a Nursing Home: Focus Group Study Among Nursing Students.

Nursing students' learning during clinical practice is largely influenced by the quality of the guidance they receive from their nurse preceptors. Students that have attended placement in nursing home settings have called for more time with nurse preceptors and an opportunity for more help from the nurses for reflection and developing critical thinking skills. To strengthen students' guidance and assessment and enhance students' learning in the practice setting, it has also been recommended to improve the collaboration between faculties and nurse preceptors. This study explores first-year nursing students' experiences of using the Technology-Optimized Practice Process in Nursing (TOPP-N) application in 4 nursing homes in Norway. TOPP-N was developed to support guidance and assessment in clinical practice in nursing education. Four focus groups were conducted with 19 nursing students from 2 university campuses in Norway. The data collection and directed content analysis were based on DeLone and McLean's information system success model. Some participants had difficulties learning to use the TOPP-N tool, particularly those who had not attended the 1-hour digital course. Furthermore, participants remarked that the content of the TOPP-N guidance module could be better adjusted to the current clinical placement, level of education, and individual achievements to be more usable. Despite this, most participants liked the TOPP-N application's concept. Using the TOPP-N mobile app for guidance and assessment was found to be very flexible. The frequency and ways of using the application varied among the participants. Most participants perceived that the use of TOPP-N facilitated awareness of learning objectives and enabled continuous reflection and feedback from nurse preceptors. However, the findings indicate that the TOPP-N application's perceived usefulness was highly dependent on the preparedness and use of the app among nurse preceptors (or absence thereof). This study offers information about critical success factors perceived by nursing students related to the use of the TOPP-N application. To develop similar learning management systems that are usable and efficient, developers should focus on personalizing the content, clarifying procedures for use, and enhancing the training and motivation of users, that is, students, nurse preceptors, and educators.

Dual cross-enhancement network for highly accurate dichotomous image segmentation

The existing image segmentation tasks mainly focus on segmenting objects with specific characteristics, such as salient, camouflaged, and meticulous objects, etc. However, the research of highly accurate Dichotomous Image Segmentation (DIS) combining these tasks has just started and still faces problems such as insufficient information interaction between layers and incomplete integration of high-level semantic information and low-level detailed features. In this paper, a new dual cross-enhancement network (DCENet) for highly accurate DIS is proposed, which mainly consists of two new modules: a cross-scaling guidance (CSG) module and a semantic cross-transplantation (SCT) module. Specifically, the CSG module adopts the adjacent-layer cross-scaling guidance method, which can efficiently interact with the multi-scale features of the adjacent layers extracted; the SCT module uses dual-branch features to complement each other. Moreover, in the way of transplantation, the high-level semantic information of the low-resolution branch is used to guide the low-level detail features of the high-resolution branch, and the features of different resolution branches are effectively fused. Finally, experimental results on the challenging DIS5K benchmark dataset show that the proposed network outperforms the 9 state-of-the-art (SOTA) networks in 5 widely used evaluation metrics. In addition, the ablation experiments also demonstrate the effectiveness of the cross-scaling guidance module and the semantic cross-transplantation module.

Physics-guided interpretable CNN for SAR target recognition

Deep Learning (DL) model has been widely used in the field of Synthetic Aperture Radar Automatic Target Recognition (SAR-ATR) and has achieved excellent performance. However, the black-box nature of DL models has been the focus of criticism, especially in the application of SAR-ATR, which is closely associated with the national defense and security domain. To address these issues, a new interpretable recognition model Physics-Guided BagNet (PGBN) is proposed in this article. The model adopts an interpretable convolutional neural network framework and uses time–frequency analysis to extract physical scattering features in SAR images. Based on the physical scattering features, an unsupervised segmentation method is proposed to distinguish targets from the background in SAR images. On the basis of the segmentation result, a structure is designed, which constrains the model’s spatial attention to focus more on the targets themselves rather than the background, thereby making the model’s decision-making more in line with physical principles. In contrast to previous interpretable research methods, this model combines interpretable structure with physical interpretability, further reducing the model’s risk of error recognition. Experiments on the MSTAR dataset verify that the PGBN model exhibits excellent interpretability and recognition performance, and comparative experiments with heatmaps indicate that the physical feature guidance module presented in this article can constrain the model to focus more on the target itself rather than the background.

A multimodal dual-branch fusion network for fetal hypoxia detection

Labor is the most severe test of a fetus’s ability to survive hypoxia. Even low-risk full-term fetuses may endure variable degrees of hypoxia as a result of intrapartum uterine contractions. It is imperative for clinicians to expeditiously identify fetal hypoxia and implement timely interventions. However, due to the intricacy and intra-class variability of fetal heart rate (FHR) data, distinguishing normal babies from acidotic fetuses is extremely challenging. This research proposes a novel multimodal dual-branch fusion network to improve the accuracy of fetal hypoxia identification. The dual-branch network is constructed through the signal slicing method. Additionally, we present a novel attention guidance module that leverages spatial attention to capture hypoxia-related information from two branches of signal. Ultimately, the network integrates maternal electronic medical records, FHR features, and FHR signals to achieve multimodal fusion of the input features. Furthermore, label smoothing can yield better calibrated networks, thus improving classification performance even further. On the public dataset, this method obtained a sensitivity of 72.58 %, a specificity of 71.08 %, a quality index of 71.59 %, and an area under the curve of 74.70 %. This method combines maternal-fetal medicine and artificial intelligence, represents a new strategy to recognize fetal acidosis.

Vessel-promoted OCT to OCTA image translation by heuristic contextual constraints

Optical Coherence Tomography Angiography (OCTA) is a crucial tool in the clinical screening of retinal diseases, allowing for accurate 3D imaging of blood vessels through non-invasive scanning. However, the hardware-based approach for acquiring OCTA images presents challenges due to the need for specialized sensors and expensive devices. In this paper, we introduce a novel method called TransPro, which can translate the readily available 3D Optical Coherence Tomography (OCT) images into 3D OCTA images without requiring any additional hardware modifications. Our TransPro method is primarily driven by two novel ideas that have been overlooked by prior work. The first idea is derived from a critical observation that the OCTA projection map is generated by averaging pixel values from its corresponding B-scans along the Z-axis. Hence, we introduce a hybrid architecture incorporating a 3D adversarial generative network and a novel Heuristic Contextual Guidance (HCG) module, which effectively maintains the consistency of the generated OCTA images between 3D volumes and projection maps. The second idea is to improve the vessel quality in the translated OCTA projection maps. As a result, we propose a novel Vessel Promoted Guidance (VPG) module to enhance the attention of network on retinal vessels. Experimental results on two datasets demonstrate that our TransPro outperforms state-of-the-art approaches, with relative improvements around 11.4% in MAE, 2.7% in PSNR, 2% in SSIM, 40% in VDE, and 9.1% in VDC compared to the baseline method. The code is available at: https://github.com/ustlsh/TransPro.

Hierarchical visual semantic guidance for enhanced relationship recognition in domain knowledge graphs

Multi-modal entity relationship recognition is a crucial foundation for constructing accurate and comprehensive domain knowledge graphs. However, due to the sparse and diverse nature of textual semantic information and the semantic differences exhibited by multi-modal data, existing methods often suffer from issues such as information loss and noise features. In this paper, the Hierarchical Visual Semantic Guidance (HVSG) Network is proposed, which utilizes images to furnish visual semantic information for text, facilitating the identification of implicit relationships between entities. Specifically, salient local instance objects are initially extracted from the global image, followed by the use of a hierarchical visual semantic construction module to establish multi-level visual semantics. The integration of multi-level visual and textual features is then achieved through a visual semantic guidance module. This work constructed a Chinese multi-modal entity relation classification dataset in the domain of unmanned vessels and conducted experiments on two datasets. Compared to state-of-the-art (SOTA) models, HVSG achieved the best performance, with F1 scores improved by 1.05% and 0.58% respectively, indicating that HVSG can offer better performance in exploring implicit relationships between textual entities.

FSPDD: A double-branch attention guided network for few-shot PCB defect detection

AbstractDuring the production of printed circuit board (PCB), there will be defects due to inappropriate operations, which will affect the use of electronic products. Majority defect detection methods cost a large number of annotated samples to train detection models. However, PCB defect samples are difficult to collect. Moreover, existing few-shot object detection methods tend to extracting low-level features from support and query images via the shared backbone such as ResNet-50. However, it is not sufficient to obtain fine-grained prior guidance. To address the above issues, we propose a few-shot PCB defect detection model with double-branch attention. Specifically, the joint attention enhancement (JAE) module is proposed to fully mine effective information of query PCB images in multiple dimensions to enhance the representation of latent defects. Then, the multi-scale guidance (MSG) module is proposed to integrate prior knowledge within support PCB images into vectors to reweight query PCB images. Experiments on the PCB defect dataset demonstrate that AP of FSPDD outperforms state-of-the-art methods under different shot settings (k=1,2,3,5,10,30) and our proposed FSPDD has a good generalization ability, in which AP reachs 0.273 when $$k=30$$ k = 30 and is 5.28% higher than SOTA methods.

Discriminative Feature Enhancement Network for few-shot classification and beyond

Few-shot classification aims to recognize query samples from novel classes given scarce labeled data, which remains a challenging problem in machine learning. This paper proposes a Discriminative Feature Enhancement Network (DFENet) to distinguish the discriminative feature of a novel category with the following characteristics: (1) a Cross-Modal Guidance Module (CMG-Module) is proposed to enrich the query vectors via leveraging label information from the additional modality; (2) a Neural-Decoding based Attention Module (NDA-Module) further explores the relationship between the query and support samples, and the attention weights depend on contribution of the support to the reconstructed query. The main intuition of the NDA-Module is to deepen our understanding of the self-attention mechanism from a neural decoding perspective, emphasizing that the best reconstruction can be used as a universal guideline; (3) a flexible triplet loss is designed to incorporate the semantic context among all classes and distinguish samples from different classes. The experimental results on two few-shot classification datasets show that our DFENet can learn a more discriminative representation for novel classes progressively. We also test the proposed method on image retrieval and facial expression recognition tasks, demonstrating consistent improvement.

SiamDCFF: Dynamic Cascade Feature Fusion for Vision Tracking.

Establishing an accurate and robust feature fusion mechanism is key to enhancing the tracking performance of single-object trackers based on a Siamese network. However, the output features of the depth-wise cross-correlation feature fusion module in fully convolutional trackers based on Siamese networks cannot establish global dependencies on the feature maps of a search area. This paper proposes a dynamic cascade feature fusion (DCFF) module by introducing a local feature guidance (LFG) module and dynamic attention modules (DAMs) after the depth-wise cross-correlation module to enhance the global dependency modeling capability during the feature fusion process. In this paper, a set of verification experiments is designed to investigate whether establishing global dependencies for the features output by the depth-wise cross-correlation operation can significantly improve the performance of fully convolutional trackers based on a Siamese network, providing experimental support for rational design of the structure of a dynamic cascade feature fusion module. Secondly, we integrate the dynamic cascade feature fusion module into the tracking framework based on a Siamese network, propose SiamDCFF, and evaluate it using public datasets. Compared with the baseline model, SiamDCFF demonstrated significant improvements.

Stereo-RIVO: Stereo-Robust Indirect Visual Odometry

Mobile robots and autonomous systems rely on advanced guidance modules which often incorporate cameras to enable key functionalities. These modules are equipped with visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms that work by analyzing changes between successive frames captured by cameras. VO/VSLAM-based systems are critical backbones for autonomous vehicles, virtual reality, structure from motion, and other robotic operations. VO/VSLAM systems encounter difficulties when implementing real-time applications in outdoor environments with restricted hardware and software platforms. While many VO systems target achieving high accuracy and speed, they often exhibit high degree of complexity and limited robustness. To overcome these challenges, this paper aims to propose a new VO system called Stereo-RIVO that balances accuracy, speed, and computational cost. Furthermore, this algorithm is based on a new data association module which consists of two primary components: a scene-matching process that achieves exceptional precision without feature extraction and a key-frame detection technique based on a model of scene movement. The performance of this proposed VO system has been tested extensively for all sequences of KITTI and UTIAS datasets for analyzing efficiency for outdoor dynamic and indoor static environments, respectively. The results of these tests indicate that the proposed Stereo-RIVO outperforms other state-of-the-art methods in terms of robustness, accuracy, and speed. Our implementation code of stereo-RIVO is available at: https://github.com/salehierfan/Stereo-RIVO.