High-level Context Information Research Articles

Efficient colorectal polyp segmentation using wavelet transformation and AdaptUNet: A hybrid U-Net

The prevalence of colorectal cancer, primarily emerging from polyps, underscores the importance of their early detection in colonoscopy images. Due to the inherent complexity and variability of polyp appearances, the task stands difficult despite recent advances in medical technology. To tackle these challenges, a deep learning model featuring a customized U-Net architecture, AdaptUNet is proposed. Attention mechanisms and skip connections facilitate the effective combination of low-level details and high-level contextual information for accurate polyp segmentation. Further, wavelet transformations are used to extract useful features overlooked in conventional image processing. The model achieves benchmark results with a Dice coefficient of 0.9104, an Intersection over Union (IoU) coefficient of 0.8368, and a Balanced Accuracy of 0.9880 on the CVC-300 dataset. Additionally, it shows exceptional performance on other datasets, including Kvasir-SEG and Etis-LaribDB. Training was performed using the Hyper Kvasir segmented images dataset, further evidencing the model's ability to handle diverse data inputs. The proposed method offers a comprehensive and efficient implementation for polyp detection without compromising performance, thus promising an improved precision and reduction in manual labour for colorectal polyp detection.

FRD-Net: a full-resolution dilated convolution network for retinal vessel segmentation

Accurate and automated retinal vessel segmentation is essential for performing diagnosis and surgical planning of retinal diseases. However, conventional U-shaped networks often suffer from segmentation errors when dealing with fine and low-contrast blood vessels due to the loss of continuous resolution in the encoding stage and the inability to recover the lost information in the decoding stage. To address this issue, this paper introduces an effective full-resolution retinal vessel segmentation network, namely FRD-Net, which consists of two core components: the backbone network and the multi-scale feature fusion module (MFFM). The backbone network achieves horizontal and vertical expansion through the interaction mechanism of multi-resolution dilated convolutions while preserving the complete image resolution. In the backbone network, the effective application of dilated convolutions with varying dilation rates, coupled with the utilization of dilated residual modules for integrating multi-scale feature maps from adjacent stages, facilitates continuous learning of multi-scale features to enhance high-level contextual information. Moreover, MFFM further enhances segmentation by fusing deeper multi-scale features with the original image, facilitating edge detail recovery for accurate vessel segmentation. In tests on multiple classical datasets,compared to state-of-the-art segmentation algorithms, FRD-Net achieves superior performance and generalization with fewer model parameters.

CHICOT: A Developer-Assistance Toolkit for Code Search with High-Level Contextual Information

We propose a source code search system named CHICOT (Code search with HIgh level COnText) to assist developers in reusing existing code. While previous studies have examined code search on the basis of code-level, fine-grained specifications such as functionality, logic, or implementation, CHICOT addresses a unique mission: code search with high-level contextual information, such as the purpose or domain of a developer's project. It achieves this feature by first extracting the context information from codebases and then considering this context during the search. It provides a VSCode plugin for daily coding assistance, and the built-in crawler ensures up-to-date code suggestions. The case study attests to the utility of CHICOT in real-world scenarios.

A deep feature fusion network with global context and cross-dimensional dependencies for classification of mild cognitive impairment from brain MRI

Background and objectives:The accurate identification of people with Mild Cognitive Impairment (MCI) who may develop Alzheimer's disease (AD) holds significant importance in facilitating timely intervention and treatment. However, current classification methods have yet to be effective due to the subtle nature of the features involved. This research aims to improve the performance of the MCI classification by enhancing the feature representation in brain MRI. MethodsWe propose an integrated model that combines Group Shuffle Depth-wise Convolution (GSDW), Global Context Network (GCN), Hybrid Multi-Focus Attention Block (HMAB), and EfficientNet-B0 architecture for MCI classification. This model extracts fine-grained features, contextual information, and long-range dependencies and aggregates low-level details with high-level context information to learn discriminative features. ResultsOur comprehensive evaluation demonstrates significant improvements in the classification performance for both progressive MCI (pMCI) and stable MCI (sMCI), surpassing existing approaches. The proposed model achieved a notable accuracy of 77.2%. ConclusionsThis study introduces a novel feature fusion technique that combines global contextual representations and cross-dimensional dependencies to enhance classification results. These findings highlight the potential of the proposed framework for early identification and intervention in individuals at risk of cognitive decline.

Multi-branch residual image semantic segmentation combined with inverse weight gated-control

The loss of pixel-level information in the multi-class segmentation task based on the U-net model results in unclear boundaries and low semantic segmentation accuracy. Aiming at this, a deep multi-branch residual Unet (IWG-MRUN) with fused inverse weight gated-control is proposed to improve the quality of image semantic segmentation. Specifically, we first introduce a deep multi-branch residual module, which used parallel convolution mode to capture the contextual feature to extract the detailed features of the input image at a deeper level. Then, we adopt an inverse weight gated-control module to enhance the diversity of up-sampling information by counterclockwise transmitting attention horizontally to improve the restoration accuracy of up-sampled image pixels. Finally, to obtain finer granularity features from low spatial resolution images, we adopt the different receptive field pyramid attention mechanisms at the highest level of the U-shaped encoder to capture high-level context information at different scales, thereby improving the accuracy of semantic segmentation. The experimental results show that the segmentation accuracy of the proposed algorithm reaches 91.80% and the CCE loss is reduced to 0.21. When compared to the Unet, BiSeNet, DeeplabV3+ and U-net + BLR model, the pixel accuracy of semantic segmentation is improved by 15.0%, 1.98%, 0.9% and 6.5%, respectively. The semantic segmentation model proposed in this paper provides an end-to-end semantic segmentation capability with the enriched finer granularity features of the target boundary and realizes the accurate segmentation of the objects in different categories.

Cross-stage recurrent feature sharing network for video dehazing

Hazy weather conditions feed serious haze effects, such as color and visibility degradations in the captured videos. Extensive research on video dehazing has been carried out. However, they lack to balance the spatial and high-level contextual information during restoration. Being a complex task in this paper, we propose a cross-stage recurrent feature sharing network to balance the competing objective. In the proposed network, a cross-stage feature merging module operates on learned features of the previous video frame using the proposed encoder decoder based generator architecture. Learned features fed into the same encoder decoder framework for the current video frame feature learning. This module benefit the network while restoring the current video frame. It shares the feature information between two consecutive frames as they have a very minute information change. Meanwhile, the design of our encoder decoder generator architecture is based on a multi-receptive resolution module, feature fusion module, and pixel-wise spatial attention module. These modules mainly focus on acquiring broad contextual information, fusing intermediate-stage features to increase the learning ability of the network, and capturing robust features while discarding unnecessary features. Experimental evaluation results and ablation study show that the proposed network is superior to existing state-of-the-art methods for video dehazing.

Inception-based global context attention network for the classification of coffee leaf diseases

Coffee is a significant global commodity that is consumed in large quantities on a daily basis, and it ranks as the second-most important product in global trade. The leaves of coffee plants are vulnerable to fungal and pest attacks that can harm their photosynthetic regions, leading to the production of low-quality, diseased beans. Accurate diagnosis of coffee disease is crucial in determining suitable remedial action. Different pathogens require different pesticides or other forms of treatment, and a precise diagnosis can guarantee that the necessary steps are taken to prevent further damage to the plant. The major objective of this research is to develop a disease detection approach for coffee leaves based on deep learning techniques. The proposed network is developed with inception modules, a global context module, and a multi-head attention module to achieve precise classification. Through the simultaneous application of different filter sizes and pooling operations on the input, inception modules facilitate the proposed network in extracting features at multiple scales and acquiring significant feature maps at different levels of abstraction. The Global Context Block employs a channel attention mechanism to generate a single feature vector that modulates the input feature maps to obtain high-level contextual information. Finally, the multi-head attention module captures complex relationships between features from different subsets and aggregates them to form a more powerful representation of the input. The experimental results indicated that the proposed network, which was trained on the BRACOL dataset, outperformed existing networks in detecting coffee leaf disease. It achieved an accuracy of 98.57% and a F1 score of 98.55%.

TopicFM: Robust and Interpretable Topic-Assisted Feature Matching

This study addresses an image-matching problem in challenging cases, such as large scene variations or textureless scenes. To gain robustness to such situations, most previous studies have attempted to encode the global contexts of a scene via graph neural networks or transformers. However, these contexts do not explicitly represent high-level contextual information, such as structural shapes or semantic instances; therefore, the encoded features are still not sufficiently discriminative in challenging scenes. We propose a novel image-matching method that applies a topic-modeling strategy to encode high-level contexts in images. The proposed method trains latent semantic instances called topics. It explicitly models an image as a multinomial distribution of topics, and then performs probabilistic feature matching. This approach improves the robustness of matching by focusing on the same semantic areas between the images. In addition, the inferred topics provide interpretability for matching the results, making our method explainable. Extensive experiments on outdoor and indoor datasets show that our method outperforms other state-of-the-art methods, particularly in challenging cases.

Automated inter-patient arrhythmia classification with dual attention neural network

Background and objectivesArrhythmia classification based on electrocardiograms (ECG) can enhance clinical diagnostic efficiency. However, due to the significant differences in the number of different categories of heartbeats, the performance of classes with fewer samples in arrhythmia classification have not met expectations under the inter-patient paradigm. This paper aims to mitigate the adverse effects of category imbalance and improve arrhythmia classification performance. MethodsWe constructed a novel dual attention hybrid network (DA-Net) for arrhythmia classification under sample imbalance, based on modified convolutional networks with channel attention (MCC-Net) and sequence-to-sequence network with global attention (Seq2Seq). The refined local features of the input heartbeat are first extracted by MCC-Net and then sent to Seq2Seq for further feature fusion. By applying local and global attention in the feature extraction and fusion parts, respectively, the method fully fuses low-level feature details and high-level context information and enhances the ability to extract discriminative features. ResultsBased on the MIT-BIH arrhythmia database, under the inter-patient paradigm without any data augmentation methods, the proposed method achieved 99.98% accuracy (ACC) for five categories. The various performance indicators are as follows: Class N: sensitivity (SEN) = 99.96%, specificity (SPEC) = 99.93%, positive predictive value (PPV) = 99.99%; Class S: SEN = 99.67%, SPEC = 99.98%, PPV = 99.56%; Class V: SEN = 100%, SPEC = 99.99%, PPV = 99.91%; Class F: SEN = 100%, PPV = 99.98%, SPEC = 97.17%. In further experiments simulating extreme cases, the model still achieved ACC of 99.54% and 98.91% in the three-category and five-category categories when the training sample size was much smaller than the test sample. ConclusionsWithout any data augmentation methods, the proposed model not only alleviates the negative impact of class imbalance and achieves excellent performance in all categories but also provides a new approach for dealing with class imbalance in arrhythmia classification. Additionally, our method demonstrates potential in conditions with fewer samples.

Context Attention Fusion Network for crowd counting

Context information, which plays a crucial role in many computer vision tasks, can benefit deep networks to construct better cognitive competence from more comprehensive surrounding. However, most existing crowd counting methods often overlook the importance of context information extracted from both global and local views. To address or alleviate this problem, this paper proposes a novel Context Attention Fusion Network, which is abbreviated as CAFNet for crowd counting. The core idea behind CAFNet is the interaction of multiple context information, including local context, cross-level context, and cross-layer context. To explore local context, we design a local context aggregation module to extract hierarchically local semantic information and then integrate them adaptively. To utilize cross-level context, a guidance attention fusion module is designed to fuse low-level feature map as the guidance of high-level context information so that the spatial details can be effectively compensated. To make full use of cross-layer features, a multi-layer context fusion module is developed to exchange the potentiality of multi-layer information to generate a high-resolution density map. Experimental results on four challenging datasets manifest that the newly proposed CAFNet can deliver impressive results compared with other state-of-the-art crowd counting models.

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

In natural language processing, short-text semantic similarity (STSS) is a very prominent field. It has a significant impact on a broad range of applications, such as question–answering systems, information retrieval, entity recognition, text analytics, sentiment classification, and so on. Despite their widespread use, many traditional machine learning techniques are incapable of identifying the semantics of short text. Traditional methods are based on ontologies, knowledge graphs, and corpus-based methods. The performance of these methods is influenced by the manually defined rules. Applying such measures is still difficult, since it poses various semantic challenges. In the existing literature, the most recent advances in short-text semantic similarity (STSS) research are not included. This study presents the systematic literature review (SLR) with the aim to (i) explain short sentence barriers in semantic similarity, (ii) identify the most appropriate standard deep learning techniques for the semantics of a short text, (iii) classify the language models that produce high-level contextual semantic information, (iv) determine appropriate datasets that are only intended for short text, and (v) highlight research challenges and proposed future improvements. To the best of our knowledge, we have provided an in-depth, comprehensive, and systematic review of short text semantic similarity trends, which will assist the researchers to reuse and enhance the semantic information.

MuTrans: Multiple Transformers for Fusing Feature Pyramid on 2D and 3D Object Detection.

One of the major components of the neural network, the feature pyramid plays a vital part in perception tasks, like object detection in autonomous driving. But it is a challenge to fuse multi-level and multi-sensor feature pyramids for object detection. This paper proposes a simple yet effective framework named MuTrans (Multiple Transformers) to fuse feature pyramid in single-stream 2D detector or two-stream 3D detector. The MuTrans based on encoder-decoder focuses on the significant features via multiple Transformers. MuTrans encoder uses three innovative self-attention mechanisms: Spatial-wise BoxAlign attention (SB) for low-level spatial locations, Context-wise Affinity attention (CA) for high-level context information, and high-level attention for multi-level features. Then MuTrans decoder processes these significant proposals including the RoI and context affinity. Besides, the Low and High-level Fusion (LHF) in the encoder reduces the number of computational parameters. And the Pre-LN [1] is utilized to accelerate the training convergence. LHF and Pre-LN are proven to reduce self-attention's computational complexity and slow training convergence. Our result demonstrates the higher detection accuracy of MuTrans than that of the baseline method, particularly in small object detection. MuTrans demonstrates a 2.1 higher detection accuracy on APS index in small object detection on MS-COCO 2017 with ResNeXt-101 backbone, a 2.18 higher 3D detection accuracy (moderate difficulty) for small object-pedestrian on KITTI, and 6.85 higher RC index (Town05 Long) on CARLA urban driving simulator platform.

Multi-stage convolutional autoencoder network for hyperspectral unmixing

Hyperspectral unmixing (HU) is a fundamental and critical task in various hyperspectral image (HSI) applications. Over the past few years, the linear mixing model (LMM) has received widely attention for its high efficiency, definite physical meaning, and being amenable to mathematical treatment. Among the various linear unmixing methods, the autoencoder unmixing network has achieved superior performance and presented more significant potential because of the powerful data fitting ability and deep feature acquisition. However, the autoencoder unmixing network, focusing on the pixel-level associations, ignores the overall distribution and long-range dependencies of materials. Inspired by the receptive field mechanism and the effectiveness of multi-stage framework, we propose a multi-stage convolutional autoencoder network for hyperspectral linear unmixing, called MSNet. MSNet is capable of learning broad contextual information without losing the detailed features by the progressively multi-stage unmixing network in the unmixing process. Compared with the conventional single-stage unmixing methods, the multi-stage framework is more robust in solving the ill-posed unmixing problem. The proposed MSNet performs more effectively and competitively than state-of-the-art algorithms by comparison experiments on synthetic and real hyperspectral datasets. The source code is available at https://github.com/yuyang95/JAG-MSNet . • A multi-stage convolutional autoencoder network is proposed for hyperspectral unmixing. • The method is more robust in solving the ill-posed unmixing problem. • The method can incorporate global high-level contextual information without losing local details.

Full-Resolution Network and Dual-Threshold Iteration for Retinal Vessel and Coronary Angiograph Segmentation.

Vessel segmentation is critical for disease diagnosis and surgical planning. Recently, the vessel segmentation method based on deep learning has achieved outstanding performance. However, vessel segmentation remains challenging due to thin vessels with low contrast that easily lose spatial information in the traditional U-shaped segmentation network. To alleviate this problem, we propose a novel and straightforward full-resolution network (FR-UNet) that expands horizontally and vertically through a multiresolution convolution interactive mechanism while retaining full image resolution. In FR-UNet, the feature aggregation module integrates multiscale feature maps from adjacent stages to supplement high-level contextual information. The modified residual blocks continuously learn multiresolution representations to obtain a pixel-level accuracy prediction map. Moreover, we propose the dual-threshold iterative algorithm (DTI) to extract weak vessel pixels for improving vessel connectivity. The proposed method was evaluated on retinal vessel datasets (DRIVE, CHASE_DB1, and STARE) and coronary angiography datasets (DCA1 and CHUAC). The results demonstrate that FR-UNet outperforms state-of-the-art methods by achieving the highest Sen, AUC, F1, and IOU on most of the above-mentioned datasets with fewer parameters, and that DTI enhances vessel connectivity while greatly improving sensitivity. The code is available at: https://github.com/lseventeen/FR-UNet.

GCNet: Grid-like context-aware network for RGB-thermal semantic segmentation

Semantic segmentation methods can achieve satisfactory performance under poor lighting conditions by exploiting the complementary cues in RGB and thermal images. However, most methods employ straightforward fusion strategies, which may insufficiently explore complementary information and ignore cross-level information propagation, except spatial information. Further, high-level contextual information may be inadequately enhanced owing to the use of simple perceptive modules. To address these limitations, we introduce a grid-like context-aware network (GCNet) for the semantic segmentation of RGB-thermal images. We use a hybrid fusion module to integrate the complementary information across modalities while considering the propagation of fusion cues by incorporating previously fused features. Considering the significance of contextual cues in semantic segmentation, a grid-like context-aware module is designed to capture rich contextual information. A three-branch discriminator is constructed to evaluate the generated prediction maps and improve the performances of the parsing results. Experiments were performed on two RGB-thermal datasets, and the results show that the proposed network achieves state-of-the-art performance.

Autoencoding low-resolution MRI for semantically smooth interpolation of anisotropic MRI.

High-resolution medical images are beneficial for analysis but their acquisition may not always be feasible. Alternatively, high-resolution images can be created from low-resolution acquisitions using conventional upsampling methods, but such methods cannot exploit high-level contextual information contained in the images. Recently, better performing deep-learning based super-resolution methods have been introduced. However, these methods are limited by their supervised character, i.e. they require high-resolution examples for training. Instead, we propose an unsupervised deep learning semantic interpolation approach that synthesizes new intermediate slices from encoded low-resolution examples. To achieve semantically smooth interpolation in through-plane direction, the method exploits the latent space generated by autoencoders. To generate new intermediate slices, latent space encodings of two spatially adjacent slices are combined using their convex combination. Subsequently, the combined encoding is decoded to an intermediate slice. To constrain the model, a notion of semantic similarity is defined for a given dataset. For this, a new loss is introduced that exploits the spatial relationship between slices of the same volume. During training, an existing in-between slice is generated using a convex combination of its neighboring slice encodings. The method was trained and evaluated using publicly available cardiac cine, neonatal brain and adult brain MRI scans. In all evaluations, the new method produces significantly better results in terms of Structural Similarity Index Measure and Peak Signal-to-Noise Ratio (p<0.001 using one-sided Wilcoxon signed-rank test) than a cubic B-spline interpolation approach. Given the unsupervised nature of the method, high-resolution training data is not required and hence, the method can be readily applied in clinical settings.

MICAR: multi-inhabitant context-aware activity recognition in home environments

The sensor-based recognition of Activities of Daily Living (ADLs) in smart-home environments enables several important applications, including the continuous monitoring of fragile subjects in their homes for healthcare systems. The majority of the approaches in the literature assume that only one resident is living in the home. Multi-inhabitant ADLs recognition is significantly more challenging, and only a limited effort has been devoted to address this setting by the research community. One of the major open problems is called data association, which is correctly associating each environmental sensor event (e.g., the opening of a fridge door) with the inhabitant that actually triggered it. Moreover, existing multi-inhabitant approaches rely on supervised learning, assuming a high availability of labeled data. However, collecting a comprehensive training set of ADLs (especially in multiple-residents settings) is prohibitive. In this work, we propose MICAR: a novel multi-inhabitant ADLs recognition approach that combines semi-supervised learning and knowledge-based reasoning. Data association is performed by semantic reasoning, combining high-level context information (e.g., residents’ postures and semantic locations) with triggered sensor events. The personalized stream of sensor events is processed by an incremental classifier, that is initialized with a limited amount of labeled ADLs. A novel cache-based active learning strategy is adopted to continuously improve the classifier. Our results on a dataset where up to 4 subjects perform ADLs at the same time show that MICAR reliably recognizes individual and joint activities while triggering a significantly low number of active learning queries.

FineAction: A Fine-Grained Video Dataset for Temporal Action Localization

Temporal action localization (TAL) is an important and challenging problem in video understanding. However, most existing TAL benchmarks are built upon the coarse granularity of action classes, which exhibits two major limitations in this task. First, coarse-level actions can make the localization models overfit in high-level context information, and ignore the atomic action details in the video. Second, the coarse action classes often lead to the ambiguous annotations of temporal boundaries, which are inappropriate for temporal action localization. To tackle these problems, we develop a novel large-scale and fine-grained video dataset, coined as FineAction, for temporal action localization. In total, FineAction contains 103K temporal instances of 106 action categories, annotated in 17K untrimmed videos. Compared to the existing TAL datasets, our FineAction takes distinct characteristics of fine action classes with rich diversity, dense annotations of multiple instances, and co-occurring actions of different classes, which introduces new opportunities and challenges for temporal action localization. To benchmark FineAction, we systematically investigate the performance of several popular temporal localization methods on it, and deeply analyze the influence of fine-grained instances in temporal action localization. As a minor contribution, we present a simple baseline approach for handling the fine-grained action detection, which achieves an mAP of 13.17% on our FineAction. We believe that FineAction can advance research of temporal action localization and beyond. The dataset is available at https://deeperaction.github.io/datasets/fineaction.

Image quality enhancement using hybrid attention networks

Image quality enhancement aims to recover rich details from degraded images, which is applied into many fields, such as medical imaging, filming production and autonomous driving. Deep convolutional neural networks (CNNs) have enabled rapid development of image quality enhancement. However, most existing CNN-based methods lack versatility when targeting different subtasks in terms of the design of networks. Besides, they often fail to balance precise spatial representations and necessary contextual information. To deal with these problems, this paper proposes a novel unified framework for low-light image enhancement, image denoising and image super-resolution. The core of this architecture is a residual hybrid attention block (RHAB), which consists of several dynamic down-sampling modules (DDM) and hybrid attention up-sampling modules (HAUM). Specifically, multi-scale feature maps are fully interacted with each other with the help of nested subnetworks so that both high-resolution spatial details and high-level contextual information can be combined to improve the representation ability of the network. Further, a hybrid attention network (HAN) is proposed and evaluations on three separate subtasks demonstrate its good performance. Extensive experiments on the authors' synthetic dataset, a more complex version, show that the authors' method achieve better quantitative and visual results compared to other state-of-the-art methods.

Aggregated pyramid attention network for mass segmentation in mammograms

Intra-class inconsistency and inter-class indistinction are intractable problems that commonly exist in breast mass segmentation from mammograms. In this work, a novel deep learning segmentation model is presented to address these problems. Firstly, we propose a simple yet effective aggregated pyramid attention module (APAM) for capturing intra-class dependencies, aiming at effectively aggregating contextual dependencies from different receptive fields to reinforce feature representations. Then, a novel aggregated pyramid attention network (APANet) is developed for further releasing the limitation of both intra-class inconsistency and inter-class indistinction. The APANet can combine low-level spatial details and high-level contextual information via encoder-decoder structure for further refining semantic representations. Finally, our proposed APANet is greatly demonstrated on two public mammographic databases including the DDSM-BCRP and INbreast, separately achieving the Dice Similarity Coefficient (DSC) of 91.04% and 94.02%.