Integration Of Contextual Information Research Articles

Prompt-guided bidirectional deep fusion network for referring image segmentation

Referring image segmentation involves accurately segmenting objects based on natural language descriptions. This poses challenges due to the intricate and varied nature of language expressions, as well as the requirement to identify relevant image regions among multiple objects. Current models predominantly employ language-aware early fusion techniques, which may lead to misinterpretations of language expressions due to the lack of explicit visual guidance of the language encoder. Additionally, early fusion methods are unable to adequately leverage high-level contexts. To address these limitations, this paper introduces the Prompt-guided Bidirectional Deep Fusion Network (PBDF-Net) to enhance the fusion of language and vision modalities. In contrast to traditional unidirectional early fusion approaches, our approach employs a prompt-guided bidirectional encoder fusion (PBEF) module to promote mutual cross-modal fusion across multiple stages of the vision and language encoders. Furthermore, PBDF-Net incorporates a prompt-guided cross-modal interaction (PCI) module during the late fusion stage, facilitating a more profound integration of contextual information from both modalities, resulting in more accurate target segmentation. Comprehensive experiments conducted on the RefCOCO, RefCOCO+, G-Ref and ReferIt datasets substantiate the efficacy of our proposed method, demonstrating significant advancements in performance compared to existing approaches.

AFC-Unet: Attention-fused full-scale CNN-transformer unet for medical image segmentation

In the field of medical image segmentation, although U-Net has achieved significant achievements, it still exposes some inherent disadvantages when dealing with complex anatomical structures and small targets, such as inaccurate target localization, blurry edges, and insufficient integration of contextual information. To address these challenges, this study proposes the Attention-Fused Full-Scale CNN-Transformer Unet (AFC-Unet), aiming to effectively overcome the limitations of traditional U-Net through multi-scale feature fusion, attention mechanisms, and CNN-Transformer hybrid modules. Specifically, we adopt an encoder–decoder architecture, incorporating full-scale feature block fusion and pyramid sampling modules to enhance the model’s ability to recognize fine to overall structural features by integrating cross-level multi-scale features. We propose the Multi-feature Fusion Attention Gates (MFAG) module, which focuses on and highlights discriminative information of potential lesions and key anatomical boundaries, effectively suppressing irrelevant background interference. We design a module Convolutional Hybrid Attention Transformer (CHAT) that integrates CNN and Transformer to address the shortcomings of traditional single models in handling long-range dependencies and global context understanding. Experimental results on three datasets of different scales demonstrate that the model’s segmentation performance for medical images surpasses state-of-the-art models, showcasing high generalization ability.

Exploring emotional intelligence in artificial intelligence systems: a comprehensive analysis of emotion recognition and response mechanisms.

This study aims to dissect the current state of emotion recognition and response mechanisms in artificial intelligence (AI) systems, exploring the progress made, challenges faced, and implicit operations of integrating emotional intelligence into AI. This study utilized a comprehensive review approach to investigate the integration of emotional intelligence (EI) into artificial intelligence (AI) systems, concentrating on emotion recognition and response mechanisms. The review process entailed formulating research questions, systematically searching academic databases such as PubMed, Scopus, and Web of Science, critically evaluating relevant literature, synthesizing the data, and presenting the findings in a comprehensive format. The study highlights the advancements in emotion recognition models, including the use of deep literacy ways and multimodal data emulsion. It discusses the challenges in emotion recognition, similar to variability in mortal expressions and the need for real-time processing. The integration of contextual information and individual traits is emphasized as enhancing the understanding of mortal feelings. The study also addresses ethical enterprises, similar as sequestration and impulses in training data. The integration of emotional intelligence into AI systems presents openings to revise mortal-computer relations. Emotion recognition and response mechanisms have made significant progress, but challenges remain. Unborn exploration directions include enhancing the robustness and interpretability of emotion recognition models, exploring cross-cultural and environment-apprehensive emotion understanding, and addressing long-term emotion shadowing and adaption. By further exploring emotional intelligence in AI systems, further compassionate and responsive machines can be developed, enabling deeper emotional connections with humans.

Improved diabetic retinopathy severity classification using squeeze-and-excitation and sparse light weight multi-level attention u-net with transfer learning from xception.

Diabetic Retinopathy (DR) is a significant cause of vision loss in diabetic patients, making early detection and accurate severity classification essential for effective management and prevention. This study aims to develop an enhanced DR severity classification approach using advanced model architectures and transfer learning to improve diagnostic accuracy and support better patient care. We propose a novel model, Xception Squeeze-and-Excitation Sparse Lightweight Multi-Level Attention U-Net (XceSE_SparseLwMLA-UNet), designed to classify DR severity using fundus images from the Messidor 1 and Messidor 2 datasets. The XceSE_SparseLwMLA-UNet integrates several advanced mechanisms: the Squeeze-and-Excitation (SE) mechanism for adaptive feature recalibration, the Sparse Lightweight Multi-Level Attention (SparseLwMLA) mechanism for effective contextual information integration, and transfer learning from the Xception architecture to enhance feature extraction capabilities. The SE mechanism refines channel-wise feature responses, while SparseLwMLA enhances the model's ability to identify complex DR patterns. Transfer learning utilizes pre-trained weights from Xception to improve generalization across DR severity levels. The proposed XceSE_SparseLwMLA-UNet model demonstrates superior performance in DR severity classification, achieving higher accuracy and improved multi-class F1 scores compared to existing models. The model's color-coded segmentation outputs offer interpretable visual representations, aiding medical professionals in assessing DR severity levels. The XceSE_SparseLwMLA-UNet model shows promise for advancing early DR diagnosis and management by enhancing classification accuracy and providing valuable visual insights. Its integration of advanced architectural features and transfer learning contributes to better patient care and improved visual health outcomes.

S[formula omitted]T-Net: A novel electroencephalogram signals-oriented emotion recognition model

In this paper, a novel skipping spatial–spectral–temporal network (S3T-Net) is developed to handle intra-individual differences in electroencephalogram (EEG) signals for accurate, robust, and generalized emotion recognition. In particular, aiming at the 4D features extracted from the raw EEG signals, a multi-branch architecture is proposed to learn spatial–spectral cross-domain representations, which benefits enhancing the model generalization ability. Time dependency among different spatial–spectral features is further captured via a bi-directional long-short term memory module, which employs an attention mechanism to integrate context information. Moreover, a skip-change unit is designed to add another auxiliary pathway for updating model parameters, which alleviates the vanishing gradient problem in complex spatial–temporal network. Evaluation results show that the proposed S3T-Net outperforms other advanced models in terms of the emotion recognition accuracy, which yields an performance improvement of 0.23% , 0.13%, and 0.43% as compared to the sub-optimal model in three test scenes, respectively. In addition, the effectiveness and superiority of the key components of S3T-Net are demonstrated from various experiments. As a reliable and competent emotion recognition model, the proposed S3T-Net contributes to the development of intelligent sentiment analysis in human–computer interaction (HCI) realm.

Recent Trends in Machine and Deep Learning for Verbal and Non-verbal Emotion Detection

Abstract: Emotion recognition, both verbal and non-verbal, is a crucial component of artificial intelligence, psychology, and human-computer interaction. Emotion recognition is an integral component that significantly contributes to the improvement of communication and interaction. The research endeavors to conduct a thorough analysis and synthesis of the most recent developments in Deep Learning (DL) and Machine Learning (ML) techniques. Specifically, the study concentrates on the recognition of both verbal and non-verbal emotions. In contrast to previous research concentrated on verbal or non-verbal emotion detection separately, the study attempts to reconcile the gap between the two by demonstrating how ML and DL can be utilized effectively to detect emotions. The study also examines new methods, including multimodal data and integration of contextual information. Additionally, the research has examined the ethical implications and difficulties associated with emotion detection technologies. Findings have also revealed the wide-reaching implications for various sectors, including healthcare, education, customer service, and entertainment, where comprehending human emotions plays a crucial role in enhancing user experience and outcomes. In conclusion, the study provides invaluable knowledge to practitioners and researchers, which may facilitate the development of more advanced and accurate systems.

SPEECH RECOGNITION SYSTEM

Speech recognition technology has witnessed remarkable progress in recent years, fueled by advancements in machine learning, deep neural networks, and signal processing techniques. This paper presents a comprehensive review of the current state-of-the-art in speech recognition systems, highlighting key methodologies and breakthroughs that have contributed to their improved performance. The paper explores various aspects, including acoustic modeling, language modeling, and the integration of contextual information, shedding light on the challenges faced and innovative solutions proposed in the field. Furthermore, the paper discusses the impact of large-scale datasets and transfer learning on the robustness and adaptability of speech recognition models. It delves into recent developments in end-to-end models and their potential to simplify the architecture while enhancing accuracy. The integration of real-time and edge computing for speech recognition applications is also explored, emphasizing the implications for practical implementations in diverse domains such as healthcare, telecommunications, and smart devices. In addition to reviewing the current landscape, the paper provides insights into future prospects and emerging trends in speech recognition research. The role of multimodal approaches, incorporating visual and contextual cues, is discussed as a potential avenue for further improvement. Ethical considerations related to privacy and bias in speech recognition systems are also addressed, emphasizing the importance of responsible development and deployment. By synthesizing current research findings and anticipating future directions, this paper contributes to the evolving discourse on speech recognition technologies, providing a valuable resource for researchers, practitioners, and industry professionals in the field. Key Words: Real-time processing , Machine learning , Deep neural networks , Technology advancements , Contextual information , Large-scale datasets Transfer learning , End-to-end models , Real-time processing Edge computing , Multimodal approaches Ethical considerations , Privacy , Bias , Future prospects Research review.

DSG-GAN:A dual-stage-generator-based GAN for cross-modality synthesis from PET to CT

PET/CT devices typically use CT images for PET attenuation correction, leading to additional radiation exposure. Alternatively, in a standalone PET imaging system, attenuation and scatter correction cannot be performed due to the absence of CT images. Therefore, it is necessary to explore methods for generating pseudo-CT images from PET images. However, traditional PET-to-CT synthesis models encounter conflicts in multi-objective optimization, leading to disparities between synthetic and real images in overall structure and texture. To address this issue, we propose a staged image generation model. Firstly, we construct a dual-stage generator, which synthesizes the overall structure and texture details of images by decomposing optimization objectives and employing multiple loss functions constraints. Additionally, in each generator, we employ improved deep perceptual skip connections, which utilize cross-layer information interaction and deep perceptual selection to effectively and selectively leverage multi-level deep information and avoid interference from redundant information. Finally, we construct a context-aware local discriminator, which integrates context information and extracts local features to generate fine local details of images and reasonably maintain the overall coherence of the images. Experimental results demonstrate that our approach outperforms other methods, with SSIM, PSNR, and FID metrics reaching 0.8993, 29.6108, and 29.7489, respectively, achieving the state-of-the-art. Furthermore, we conduct visual experiments on the synthesized pseudo-CT images in terms of image structure and texture. The results indicate that the pseudo-CT images synthesized in this study are more similar to real CT images, providing accurate structure information for clinical disease analysis and lesion localization.

Composite Backbone Small Object Detection Based on Context and Multi-Scale Information with Attention Mechanism

Object detection has gained widespread application across various domains; nevertheless, small object detection still presents numerous challenges due to the inherent limitations of small objects, such as their limited resolution and susceptibility to interference from neighboring elements. To improve detection accuracy of small objects, this study presents a novel method that integrates context information, attention mechanism, and multi-scale information. First, to realize feature augmentation, a composite backbone network is employed which can jointly extract object features. On this basis, to efficiently incorporate context information and focus on key features, the composite dilated convolution and attention module (CDAM) is designed, consisting of a composite dilated convolution module (CDM) and convolutional block attention module (CBAM). Then, a feature elimination module (FEM) is introduced to reduce the feature proportion of medium and large objects on feature layers; the impact of neighboring objects on small object detection can thereby be mitigated. Experiments conducted on MS COCO validate the superior performance of the method compared with baseline detectors, while it yields an average enhancement of 0.8% in overall detection accuracy, with a notable enhancement of 2.7% in small object detection.

Chemogenetic inhibition of the ventral hippocampus but not its direct projection to the prelimbic cortex attenuates context-specific operant responding.

Previous work has demonstrated the importance of the prelimbic cortex (PL) in contextual control of operant behavior. However, the associated neural circuitry responsible for providing contextual information to the PL is not well understood. In Pavlovian fear conditioning the ventral hippocampus (vH) and its projection to the PL have been shown to be important in supporting the effects of context on learning. The present experiments used chemogenetic inhibition of the direct vH-PL projection or the vH to determine involvement in expression of context-specific operant behavior. Rats were injected with an inhibitory DREADD (hM4Di) or mCherry-only into the vH, and subsequently trained to perform a lever press response for a food pellet in a distinct context. The DREADD ligand clozapine-n-oxide (CNO) was then delivered directly into the PL (experiment 1) and then systemically (experiment 2) prior to tests of the response in the training context as well as an equally familiar but untrained context. vH (systemic CNO) but not vH-PL (intra-PL CNO) inhibition was found to attenuate operant responding in its acquisition context. A third experiment, using the same rats, showed that chemogenetic inhibition of vH also reduced Pavlovian contextual fear. The present results suggest that multisynapatic connections between the vH and PL may be responsible for integration of contextual information with operant behavior.

Semantic-specific and domain-general mechanisms for integration and update of contextual information.

Recent research has highlighted the importance of domain-general processes and brain regions for language and semantic cognition. Yet, this has been mainly observed in executively demanding tasks, leaving open the question of the contribution of domain-general processes to natural language and semantic cognition. Using fMRI, we investigated whether neural processes reflecting context integration and context update-two key aspects of naturalistic language and semantic processing-are domain-specific versus domain-general. Thus, we compared neural responses during the integration of contextual information across semantic and non-semantic tasks. Whole-brain results revealed both shared (left posterior-dorsal inferior frontal gyrus, left posterior inferior temporal gyrus, and left dorsal angular gyrus/intraparietal sulcus) and distinct (left anterior-ventral inferior frontal gyrus, left anterior ventral angular gyrus, left posterior middle temporal gyrus for semantic control only) regions involved in context integration and update. Furthermore, data-driven functional connectivity analysis clustered domain-specific versus domain-general brain regions into distinct but interacting functional neural networks. These results provide a first characterisation of the neural processes required for context-dependent integration during language processing along the domain-specificity dimension, and at the same time, they bring new insights into the role of left posterior lateral temporal cortex and left angular gyrus for semantic cognition.

Event Extraction With Dynamic Prefix Tuning and Relevance Retrieval

We consider event extraction in a generative manner with template-based conditional generation. Although there is a rising trend of casting the task of event extraction as a sequence generation problem with prompts, these generation-based methods have several significant challenges, including using suboptimal prompts, static event type information, and the overwhelming number of irrelevant event types. In this paper, we propose a generative template-based method with dynamic prefixes and a relevance retrieval framework for event extraction ( <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GREE</small> ) by first integrating context information with type-specific prefixes to learn a context-specific prefix for each context, and then retrieving the relevant event types with an adaptive threshold. Experimental results show that our model achieves competitive results with the state-of-the-art classification-based model <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OneIE</small> on ACE 2005 and achieves the best performances on ERE. Additionally, our model is proven to be portable to new types of events effectively.

Weakly supervised human skin segmentation using guidance attention mechanisms

Human skin segmentation is a crucial task in computer vision and biometric systems, yet it poses several challenges such as variability in skin colour, pose, and illumination. This paper presents a robust data-driven skin segmentation method for a single image that addresses these challenges through the integration of contextual information and efficient network design. In addition to robustness and accuracy, the integration into real-time systems requires a careful balance between computational power, speed, and performance. The proposed method incorporates two attention modules, Body Attention and Skin Attention, that utilize contextual information to improve segmentation results. These modules draw attention to the desired areas, focusing on the body boundaries and skin pixels, respectively. Additionally, an efficient network architecture is employed in the encoder part to minimize computational power while retaining high performance. To handle the issue of noisy labels in skin datasets, the proposed method uses a weakly supervised training strategy, relying on the Skin Attention module. The results of this study demonstrate that the proposed method is comparable to, or outperforms, state-of-the-art methods on benchmark datasets.

Networked Sensing for Radiation Detection, Localization, and Tracking

The detection, identification, and localization of illicit radiological and nuclear material continue to be key components of nuclear non-proliferation and nuclear security efforts around the world. Networks of radiation detectors deployed at strategic locations in urban environments have the potential to provide continuous radiological/nuclear (R/N) surveillance and provide high probabilities of intercepting threat sources. The integration of contextual information from sensors such as video, Lidar, and meteorological sensors can provide significantly enhanced situational awareness, and improved detection and localization performance through the fusion of the radiological and contextual data. In this work, we present details of our work to establish a city-scale multi-sensor network testbed for intelligent, adaptive R/N detection in urban environments, and develop new techniques that enable city-scale source detection, localization, and tracking.

SaGCN: Semantic-Aware Graph Calibration Network for Temporal Sentence Grounding

Temporal sentence grounding is a challenging task that aims to localize the semantic corresponding segment from the untrimmed video according to the given query language description. Existing methods either utilize a cross-modal matching architecture following a scan-and-rank pipeline or directly predict the probabilities of being the target boundary for each frame based on the entire video content. However, such methods are weak when some of the critical semantic concepts in the query are actually relevant to multiple video segments or the desired video segment contains a query-irrelevant scene due to ignoring query semantic concepts and local and global cross-modal context. In this paper, we propose a novel semantic-aware graph calibration network (SaGCN) to address the issues mentioned above. Specifically, we first introduce a semantic-aware local relational graph module to capture the inherent relationships among the specific semantic concept relevant local contextual information for fine-grained cross-modal information interactions. Then, a semantic-aware global relational graph module is derived for global contextual information integration and achieving cross-modal alignment. Finally, an attention-based calibration module is designed for eliminating the irrelevant information maintained in the visual modality under the guidance of query description. Extensive experiments verify the effectiveness of our proposed SaGCN on two widely used datasets (Charades-STA and TACoS), in which we achieve significant and consistent improvement compared to the state-of-the-art approaches.

Micro-expression Recognition Based on DCBAM-EfficientNet Model

To address the problems of low accuracy of existing deep learning-based micro-expression recognition models, numerous network parameters, and the difficulty of mobile deployment of micro-expression recognition models, this paper proposes DCBAM-EfficientNet, a micro-expression recognition model that uses the lightweight network EfficientNet as the backbone network and incorporates the attention module. The network can guarantee the accuracy of micro-expression recognition with relatively few network parameters. The attention mechanism allows the more expressive micro-expression features to be highlighted, and the CBAM attention is improved into a DCBAM model, where the large convolution kernel in the spatial attention module of CBAM is replaced by a dilated convolution with the same receptive field, reducing the network parameters while better preserving the spatial features of the image. The integration of the DCBAM model into the main structure of EfficientNet enables better integration of contextual information. Data enhancement is used to process the micro-expression dataset to decrease the occurrence of overfitting and improve the generalization ability of the model. The results demonstrate that the optimized model DCBAM-EfficientNet can effectively promote the recognition accuracy of micro-expressions, significantly reduce the quantity and volume of model parameters, and provide a reference for the deployment of mobile micro-expression recognition models.

Cloud Detection From High-Resolution Remote Sensing Images Based on Convolutional Neural Networks With Geographic Features and Contextual Information

The diversity and complexity of the subsurface, as well as the similarity of cloud features to those of highlighted surface objects (especially snowy areas), are the main reasons for the current limitations in cloud detection accuracy. This letter proposes a semantic segmentation network fusing geographic features with contextual information for cloud detection named GCI_CD. We design three modules: geographical-cloud information fusion module (to distinguish between clouds and snow), separable ResNet with CSAM encoder (to enhance channel and spatial association information extraction), contextual information integration module (to extract multi-scale features). To the best of our knowledge, this is the first time that geographic information has been fused in a cloud detection method. The validity of our proposed method is demonstrated using a high-resolution GF-1 dataset containing the near-infrared band and dividing the snow-covered and non-snow-covered areas. The experimental results show that GCI_CD has a better performance not only in snow-covered regions but also in the non-snow-covered regions. Especially for snow-covered images, the IOU of GCI_CD’s cloud detection results are improved by 5% on average compared to other cloud detection methods.

A novel MCF-Net: Multi-level context fusion network for 2D medical image segmentation

Medical image segmentation is a crucial step in the clinical applications for diagnosis and analysis of some diseases. U-Net-based convolution neural networks have achieved impressive performance in medical image segmentation tasks. However, the multi-level contextual information integration capability and the feature extraction ability are often insufficient. In this paper, we present a novel multi-level context fusion network (MCF-Net) to improve the performance of U-Net on various segmentation tasks by designing three modules, hybrid attention-based residual atrous convolution (HARA) module, multi-scale feature memory (MSFM) module, and multi-receptive field fusion (MRFF) module, to fuse multi-scale contextual information. HARA module was proposed to effectively extract multi-receptive field features by combing atrous spatial pyramid pooling and attention mechanism. We further design the MSFM and MRFF modules to fuse features of different levels and effectively extract contextual information. The proposed MCF-Net was evaluated on the ISIC 2018, DRIVE, BUSI, and Kvasir-SEG datasets, which have challenging images of many sizes and widely varying anatomy. The experimental results show that MCF-Net is very competitive with other U-Net models, and it offers tremendous potential as a general-purpose deep learning model for 2D medical image segmentation.

FEA-Swin: Foreground Enhancement Attention Swin Transformer Network for Accurate UAV-Based Dense Object Detection.

UAV-based object detection has recently attracted a lot of attention due to its diverse applications. Most of the existing convolution neural network based object detection models can perform well in common object detection cases. However, due to the fact that objects in UAV images are spatially distributed in a very dense manner, these methods have limited performance for UAV-based object detection. In this paper, we propose a novel transformer-based object detection model to improve the accuracy of object detection in UAV images. To detect dense objects competently, an advanced foreground enhancement attention Swin Transformer (FEA-Swin) framework is designed by integrating context information into the original backbone of a Swin Transformer. Moreover, to avoid the loss of information of small objects, an improved weighted bidirectional feature pyramid network (BiFPN) is presented by designing the skip connection operation. The proposed method aggregates feature maps from four stages and keeps abundant information of small objects. Specifically, to balance the detection accuracy and efficiency, we introduce an efficient neck of the BiFPN network by removing a redundant network layer. Experimental results on both public datasets and a self-made dataset demonstrate the performance of our method compared to the state-of-the-art methods in terms of detection accuracy.

An Attention Encoder-Decoder Network Based on Generative Adversarial Network for Remote Sensing Image Dehazing

Remote sensing image dehazing is a difficult problem for its complex characteristics. It can be regarded as the preprocessing of high-level tasks of remote sensing images. To remove haze from the hazy remote sensing image, an encoder-decoder based on generative adversarial network is proposed. It first learns the low-frequency information of the image, and then learns the high-frequency information of the image. The skip connection is also added in the network to avoid losing information. To further improve the ability of learning more useful information, a multi-scale attention module is proposed. Meanwhile, a CBlock module is also designed to extract more feature information. It can capture different size of receptive fields. In order to reduce the computational pressure of the network, a distillation module is used in the network. Inspired by multi-scale network, an enhance module is designed and introduced it in the end of the network to further improve the dehazing ability of the network by integrating context information on multi-scale. We compared with five methods and our proposed method on RICE dataset. Experimental results show that our method achieves the best effect, both qualitatively and quantitatively.