Context Information Model Research Articles

CasUNeXt: A Cascaded Transformer With Intra‐ and Inter‐Scale Information for Medical Image Segmentation

ABSTRACTDue to the Transformer's ability to capture long‐range dependencies through Self‐Attention, it has shown immense potential in medical image segmentation. However, it lacks the capability to model local relationships between pixels. Therefore, many previous approaches embedded the Transformer into the CNN encoder. However, current methods often fall short in modeling the relationships between multi‐scale features, specifically the spatial correspondence between features at different scales. This limitation can result in the ineffective capture of scale differences for each object and the loss of features for small targets. Furthermore, due to the high complexity of the Transformer, it is challenging to integrate local and global information within the same scale effectively. To address these limitations, we propose a novel backbone network called CasUNeXt, which features three appealing design elements: (1) We use the idea of cascade to redesign the way CNN and Transformer are combined to enhance modeling the unique interrelationships between multi‐scale information. (2) We design a Cascaded Scale‐wise Transformer Module capable of cross‐scale interactions. It not only strengthens feature extraction within a single scale but also models interactions between different scales. (3) We overhaul the multi‐head Channel Attention mechanism to enable it to model context information in feature maps from multiple perspectives within the channel dimension. These design features collectively enable CasUNeXt to better integrate local and global information and capture relationships between multi‐scale features, thereby improving the performance of medical image segmentation. Through experimental comparisons on various benchmark datasets, our CasUNeXt method exhibits outstanding performance in medical image segmentation tasks, surpassing the current state‐of‐the‐art methods.

HyperCARS: Using Hyperbolic Embeddings for Generating Hierarchical Contextual Situations in Context-Aware Recommender Systems

Contextual situations, such as having dinner at a restaurant on Friday with the spouse, became a useful mechanism to represent context in context-aware recommender systems (CARS). Prior research has shown important advantages of using latent embedding representation approaches to model contextual information in the Euclidean space leading to better recommendations. However, these traditional approaches have major challenges with the construction of proper embeddings of hierarchical structures of contextual information, as well as with interpretations of the obtained representations. To address these problems, we propose the HyperCARS method that models hierarchical contextual situations in the latent hyperbolic space. HyperCARS combines hyperbolic embeddings with hierarchical clustering to construct contextual situations, which allows loose coupling of the contextual modeling component with recommendation algorithms and, therefore, provides flexibility to use a broad range of previously developed recommendation algorithms. We demonstrate empirically that HyperCARS better captures and interprets hierarchical contextual representations, leading to better context-aware recommendations. Because hyperbolic embeddings can also be used in many other applications besides CARS, we also propose the latent embeddings representation framework that systematically classifies prior work on embeddings and identifies novel research streams for hyperbolic embeddings across information systems applications.

Multi-perspective feature collaborative perception learning network for non-destructive detection of pavement defects

Pavement defects detection has made significant progress with the development of convolutional neural networks. Due to the topological complexity of pavement defects regions, most existing detection methods are limited to a limited number of types of defects, which results in unsatisfactory detection results for multi-classified defects. Several studies have used enhanced data to address this issue, however, few have considered the inefficiency caused by imbalanced differences between classes from the perspective of the network model itself. In light of this issue, our goal is to fill this gap by better exploring the fusion of different feature information to improve the performance of the network. As a result, we propose a multi-perspective feature collaborative perception learning network (MFCPLNet), which uses the model's contextual information to guide high-quality multi-classified defects detection and that can be adapted to various pavement conditions. Specifically, for adaptive feature extraction, an irregular perceptual extraction network (IPENet) with enhanced edge information is first designed to draw upon hybrid deformable convolution for efficient network block hybridization. Additionally, a multi-balanced collaborative learning mechanism (MCLM) consisting of auxiliary axial information and 3D feature guidance is constructed to enhance deep semantic features, improve the localization of defects in defective regions, and compensate for weak category supervision. Lastly, in the detection head, an iterative soft aggregation module has been designed to enhance scalable spatial interactions. The results of extensive testing were based on publicly available pavement defects datasets. Based on the proposed network, the mAP index reaches 29.6% and 33.8%, respectively, an increase of 8.3% and 4.6% over the baseline, which is qualitatively and quantitatively superior to mainstream methods.

DAGCN: Dynamic and Adaptive Graph Convolutional Network for Salient Object Detection.

Deep-learning-based salient object detection (SOD) has achieved significant success in recent years. The SOD focuses on the context modeling of the scene information, and how to effectively model the context relationship in the scene is the key. However, it is difficult to build an effective context structure and model it. In this article, we propose a novel SOD method called dynamic and adaptive graph convolutional network (DAGCN) that is composed of two parts, adaptive neighborhood-wise graph convolutional network (AnwGCN) and spatially restricted K-nearest neighbors (SRKNN). The AnwGCN is novel adaptive neighborhood-wise graph convolution, which is used to model and analyze the saliency context. The SRKNN constructs the topological relationship of the saliency context by measuring the non-Euclidean spatial distance within a limited range. The proposed method constructs the context relationship as a topological graph by measuring the distance of the features in the non-Euclidean space, and conducts comparative modeling of context information through AnwGCN. The model has the ability to learn the metrics from features and can adapt to the hidden space distribution of the data. The description of the feature relationship is more accurate. Through the convolutional kernel adapted to the neighborhood, the model obtains the structure learning ability. Therefore, the graph convolution process can adapt to different graph data. Experimental results demonstrate that our solution achieves satisfactory performance on six widely used datasets and can also effectively detect camouflaged objects. Our code will be available at: https://github.com/CSIM-LUT/DAGCN.git.

CASTNet: A Context-Aware, Spatio-Temporal Dynamic Motion Prediction Ensemble for Autonomous Driving

Autonomous vehicles are cyber-physical systems that combine embedded computing and deep learning with physical systems to perceive the world, predict future states, and safely control the vehicle through changing environments. The ability of an autonomous vehicle to accurately predict the motion of other road users across a wide range of diverse scenarios is critical for both motion planning and safety. However, existing motion prediction methods do not explicitly model contextual information about the environment, which can cause significant variations in performance across diverse driving scenarios. To address this limitation, we propose CASTNet : a dynamic, context-aware approach for motion prediction that (i) identifies the current driving context using a spatio-temporal model, (ii) adapts an ensemble of motion prediction models to fit the current context, and (iii) applies novel trajectory fusion methods to combine predictions output by the ensemble. This approach enables CASTNet to improve robustness by minimizing motion prediction error across diverse driving scenarios. CASTNet is highly modular and can be used with various existing image processing backbones and motion predictors. We demonstrate how CASTNet can improve both CNN-based and graph-learning-based motion prediction approaches and conduct ablation studies on the performance, latency, and model size for various ensemble architecture choices. In addition, we propose and evaluate several attention-based spatio-temporal models for context identification and ensemble selection. We also propose a modular trajectory fusion algorithm that effectively filters, clusters, and fuses the predicted trajectories output by the ensemble. On the nuScenes dataset, our approach demonstrates more robust and consistent performance across diverse, real-world driving contexts than state-of-the-art techniques.

Attention-guided multi-granularity fusion model for video summarization

Video summarization has attracted extensive attention benefiting from its valuable capability to facilitate video browsing. While achieving notable improvement, existing methods still fail to sufficiently and effectively model contextual information within videos, hindering the summarization performance owing to a deficiency in powerful contextual representations. To address this limitation, we present a novel Attention-Guided Multi-Granularity Fusion Model (AMFM), which allows for optimizing the modeling process from the context capturing and fusion perspective. AMFM comprises three dominant components including a content-aware enhancement (CAE) module, a multi-granularity encoder (MGE), and a scale-adaptive fusion (SAF) module. More specifically, CAE dynamically enhances pre-trained visual features by learning the potential visual relationship across frame-level and video-level embeddings. Subsequently, coarse-grained and fine-grained contextual information is simultaneously modeled in the same representation space by MGE with the combination of self-attention and temporal convolution scheme. Furthermore, the multi-granularity representations with a significant difference in the semantic scale are adaptively fused by SAF. Our method can precisely pinpoint key segments by effectively modeling and processing rich temporal representations. Extensive comparisons with state-of-the-art methods on standard datasets demonstrate the effectiveness of the proposed method, and the ablation studies further verify the positive impact of each module in our model.

DER-GCN: Dialog and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialog Emotion Recognition.

With the continuous development of deep learning (DL), the task of multimodal dialog emotion recognition (MDER) has recently received extensive research attention, which is also an essential branch of DL. The MDER aims to identify the emotional information contained in different modalities, e.g., text, video, and audio, and in different dialog scenes. However, the existing research has focused on modeling contextual semantic information and dialog relations between speakers while ignoring the impact of event relations on emotion. To tackle the above issues, we propose a novel dialog and event relation-aware graph convolutional neural network (DER-GCN) for multimodal emotion recognition method. It models dialog relations between speakers and captures latent event relations information. Specifically, we construct a weighted multirelationship graph to simultaneously capture the dependencies between speakers and event relations in a dialog. Moreover, we also introduce a self-supervised masked graph autoencoder (SMGAE) to improve the fusion representation ability of features and structures. Next, we design a new multiple information Transformer (MIT) to capture the correlation between different relations, which can provide a better fuse of the multivariate information between relations. Finally, we propose a loss optimization strategy based on contrastive learning to enhance the representation learning ability of minority class features. We conduct extensive experiments on the benchmark datasets, Interactive Emotional Dyadic Motion Capture (IEMOCAP) and Multimodal EmotionLines Dataset (MELD), which verify the effectiveness of the DER-GCN model. The results demonstrate that our model significantly improves both the average accuracy and the F1 value of emotion recognition. Our code is publicly available at https://github.com/yuntaoshou/DER-GCN.

Overcoming Fuzziness with Semantic Modeling and AI

Today, customer service in mechanical engineering struggles with a wide variety of challenges when it comes to providing relevant information for specific use cases. The information requirements of service technicians, on the other hand, have increased enormously. On the one hand due to increasingly complex products, but also due to the growing shortage of trained specialists. Fast and efficient provision of relevant information is essential. Remote assistance systems and portal solutions are being used to meet these challenges. A heterogeneous information landscape, characterized by data silos, makes this difficult. In a preliminary study, the authors have developed an approach based on the formalization of knowledge in knowledge graphs that bridges the data silos and thus enables efficient information provision by linking information based on use cases. However, this approach requires high data quality and suitable metadata concepts. Inaccuracies in classification systems create unwanted fuzziness when linking information. The study based on this shows how the fuzziness can be eliminated by supplementary modeling of context information and by extended modeling methods in the form of so-called DynaRules. The authors also compare how AI methods and DynaRules can be used to check the plausibility of deep links with potential fuzziness.

Semantic Context Information Modeling With Neural Networks in Customer Order Behavior Classification

Semantic Context Information Modeling With Neural Networks in Customer Order Behavior Classification

Ecc‐RCNN: An efficient and high‐accuracy object detection framework for transmission line defect identification

AbstractIn order to improve the accuracy of image‐based transmission line defect detection, while reducing the computational complexity and the high demand on chip performance, an object detection framework is proposed, which aims to improve model performance without increasing the scale of the model and the amount of calculation. An efficient feature fusion module to combine different‐level semantic features in non‐linear transformations is introduced. It includes channel‐level hierarchy features, linear projection and residual mappings to gather task‐oriented features across different spatial locations and scales. Then a context information modelling module is proposed to extract features around the target objects, which further increases the detection accuracy. Finally, an Intersection‐over‐Union‐based training examples sampling strategy is adopted to alleviate the class imbalance problem. Experiments on dataset show that the proposed method, with a similar number of model parameters, has an accuracy improved by 8.1% compared to the baseline, and outperforms all the competitors in the area of transmission line defect detection.

Information-Enhanced Hierarchical Self-Attention Network for Multiturn Dialog Generation

Transformer structure has shown promising results in multiturn dialog generation. The self-attention mechanism can learn global dependencies but ignores local information, limiting the model's ability to model context information. In this article, we propose an information-enhanced hierarchical self-attention network (IEHSA). In the word-level encoder, words in successive windows are automatically encoded as local information, and words with dependent words are automatically encoded as syntactic information, both of which are used to enhance word information and then feed into the self-attention mechanism. In the utterance-level encoder, adjacent utterance representations are automatically encoded as dialog structure information, and the self-attention mechanism is used to update the utterance representations. The context and masked response representation are then updated using the self-attention mechanism in the decoder. Finally, the correlation between context and reply is calculated and used in further decoding. We compared IEHSA with the current popular hierarchical model on several datasets, and the experiments show that the proposed method has substantial improvements in both metric-based and human evaluations.

An attention-based multiscale transformer network for remote sensing image change detection

The bi-temporal change detection (CD) is still challenging for high-resolution optical remote sensing data analysis due to various factors such as complex textures, seasonal variations, climate changes, and new requirements. We propose an attention-based multiscale transformer network (AMTNet) that utilizes a CNN-transformer structure to address this issue. Our Siamese network based on the CNN-transformer architecture uses ConvNets as the backbone to extract multiscale features from the raw input image pair. We then employ attention and transformer modules to model contextual information in bi-temporal images effectively. Additionally, we use feature exchange to bridge the domain gap between different temporal image domains by partially exchanging features between the two Siamese branches of our AMTNet. Experimental results on four commonly used CD datasets – CLCD, HRSCD, WHU-CD, and LEVIR-CD – demonstrate the effectiveness and efficiency of our proposed AMTNet approach. The code for this work will be available on GitHub.11https://github.com/linyiyuan11/AMT_Net.

Combining CNN and transformers for full-reference and no-reference image quality assessment

Most deep learning approaches for image quality assessment use regression from deep features extracted by CNN (Convolutional Neural Networks). However, non-local information is usually neglected in existing methods. Motivated by the recent success of transformers in modeling contextual information, we propose a hybrid framework that utilizes a vision transformer backbone to extract features and a CNN decoder for quality estimation. We propose a shared feature extraction scheme for both FR and NR settings. A two-branch structured attentive quality predictor is devised for quality prediction. Evaluation experiments on various IQA datasets, including LIVE, CSIQ and TID2013, LIVE-Challenge, KADID-10 K, and KONIQ-10 K, show that our proposed models achieve outstanding performance for both FR and NR settings.

Implicit sentiment analysis based on affective knowledge and event information

In recent years, implicit sentiment analysis, which aims to detect sentiment of a sentence that don’t contain obvious sentiment words, has become an attractive research topic. This paper focuses on event-centric implicit sentiment analysis, which utilizes the sentiment-aware event to infer the sentiment polarity of the sentence. Existing event-based implicit sentiment analysis methods typically treat entities or noun phrases in the text as events, or model contextual information to indirectly infer events using sophisticated models, but these methods fail to fully capture event information. To address these issues, this paper defines events as <subject, predicate, object>. Based on this event representation, neural tensor network was used to model the interaction between event elements and extract high-level semantic features of events. In addition, a novel affective enhanced graph model was proposed to capture sentiment-related dependencies between context words. Furthermore, this paper considers the case where a sentence contains multiple events, and constructs an event-centric implicit sentiment analysis dataset, where each sentence contains at least one event triplet. Experimental results on the constructed dataset demonstrate the effectiveness of our proposed approach.

Performance analysis of sentiment classification based neural network

Deep learning has more significant advantages for word embedding technology than sentiment analysis. This paper studies the application of deep learning on the word embedding problem in context, mainly discusses the RNN model with Word2Vec and without Word2Vec, then compares and analyzes their performance in the experiment, mainly evaluating the accuracy and test loss of seven models. This paper compares and illustrates the model which gets the different results in experiments, complementing the model and re-running the model, and analyzing the reasons for the difference in the performance of each model. The seven models are a single-layer neural network, multiple-layer (two and three) feed-forward neural networks, Convolutional Neural Network (CNN)- A feedforward neural network, which consists of single or multiple convolutional layers, pooling layers, and a fully connected layer on top, so this model is good at image processing. Long Short Term Memory (LSTM)- A temporal recurrent neural network, the advantage of the model is it could solve the gradient disappearance and explosion problem when it handles the long-sequence problem. Bi-directional Long Short Term Memory (Bi-LSTM)-Composed of forwarding LSTM and backward LSTM, it is very common for sequence labelling tasks that are related to the top and bottom, which are often used to model context information in NLP. Bi-directional Encoder Representation from Transformers (BERT)- A bidirectional language model. Finally, this paper analyses and evaluates these models with a specific illustration and research.

On Understanding Context Modelling for Adaptive Authentication Systems

In many situations, it is of interest for authentication systems to adapt to context (e.g., when the user’s behavior differs from the previous behavior). Hence, representing the context with appropriate and well-designed models is crucial. We provide a comprehensive overview and analysis of research work on C ontext M odelling f or A daptive A uthentication systems (CM4AA). To this end, we pursue three goals based on the Systematic Mapping Study (SMS) and Systematic Literature Review (SLR) research methodologies. We first present a SMS to structure the research area of CM4AA ( goal 1 ). We complement the SMS with an SLR to gather and synthesise evidence about context information and its modelling for adaptive authentication systems ( goal 2 ). From the knowledge gained from goal 2, we determine the desired properties of the context information model and its use for adaptive authentication systems ( goal 3 ). Motivated to find out how to model context information for adaptive authentication, we provide a structured survey of the literature to date on CM4AA and a classification of existing proposals according to several analysis metrics. We demonstrate the ability of capturing a common set of contextual features that are relevant for adaptive authentication systems independent from the application domain. We emphasise that despite the possibility of a unified framework, no standard for CM4AA exists.

Transformer-Based Domain-Specific Representation for Unsupervised Domain Adaptive Vehicle Re-Identification

Fully-supervised vehicle re-identification (re-ID) methods are faced with performance degradation when applied to new image domains. Therefore, developing unsupervised domain adaptation (UDA) to transfer the knowledge from learned source domain to new unlabeled target domain becomes an indispensable task. It is challenging because different domains have various image appearances, such as different backgrounds, illuminations and resolutions, especially when cameras have different viewpoints. To tackle this domain gap issue, a novel Transformer-based Domain-Specific Representation learning network (TDSR) is proposed to dynamically focus on corresponding detailed hints for each domain. Specifically, with the source and target domain being trained simultaneously, a domain encoding module is proposed to introduce domain information into the network. The original features of source and target domains are enriched with these domain encodings first, and then sequentially processed by a Transformer encoder to model contextual information and a decoder to summarize the encoded features into the final domain-specific feature representations. Moreover, we propose a Contrastive Clustering Loss (CCL) to directly optimize the distribution of features at cluster level. Instances are overall pulled closer to the prototype of the same identity, and pushed farther from the prototypes of different identities. It helps compact the clusters in the latent space and improve the discriminative capability of the network, leading to more accurate pseudo-label assignment in TDSR. Our method outperforms the state-of-the-art UDA methods on vehicle re-ID benchmark datasets VeRi and VehicleID on both real-world to real-world and synthetic to real-world settings.

SPANet: Spatial Adaptive Convolution Based Content-Aware Network for Aerial Image Semantic Segmentation

Semantic segmentation of remote sensing images encounters four significant difficulties: 1) complex backgrounds, 2) large-scale differences, 3) numerous small objects, and 4) extreme foreground-background imbalance. However, the existing generic semantic segmentation models mainly focus on the modeling context information and rarely focus on these four issues. This article presents an enhanced remote sensing image semantic segmentation framework to solve these problems through the Hierarchical Atrous Pyramid module (HASP) and spatial-adaptive convolution based FPN decoder framework (SPA-FPN). On the one hand, HASP solved the problem of complex backgrounds and large-scale differences by further enlarging the receptive field of the network through the cascade of atrous convolution with various rates. On the other hand, spatial adaptive convolution is embedded in FPN decoder framework step by step to solve the problems of numerous small objects and extreme foreground-background imbalance. Besides, a boundary-based loss function is constructed to help the network optimize the relevant segmentation results. Extensive experiments over iSAID and ISPRS Vaihingen datasets reflect the superiority of the presented structure to conventional state-of-the-art semantic segmentation approaches. The code will be accessible.

Indoor AR Navigation and Emergency Evacuation System Based on Machine Learning and IoT Technologies

In order to evacuate people safely and quickly in indoor disaster environments, it is necessary to estimate the current location of the individuals, detect disaster situations, predict disaster propagation, derive optimal individual escape paths, and implement a user-friendly and intuitive guidance system. In this study, we propose a machine-learning-based indoor augmented reality (AR) navigation and emergency evacuation system that can guide an optimal escape path for individual users. To detect emergency events and deliver sensing data, an Internet of Things (IoT)-enabled ad hoc network is considered. To deliver the sensing data safely and reliably to the server, we present a hybrid reinforcement-learning-based routing algorithm that combines direct and indirect <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning methods. Prediction of disaster propagation at multiple time scales is important to prevent dangerous situations. We propose a simple disaster area prediction method that ensembles elementary component gradient boosting machine models. User location is estimated by a deep neural network using the received signal strength from beacon nodes. To derive the optimum evacuation path for each individual, we propose a novel model-based <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning method, in which we consider the building structural model and disaster context information. The performance of the proposed system is experimentally evaluated for various disaster scenarios.

Dense Traffic Detection at Highway-Railroad Grade Crossings

In the United States, highway-railroad grade crossings are easily congested, which not only causes significant traffic delays to travelers but also brings potential threats to the first responders for emergencies. Unfortunately, very limited research efforts have been dedicated to developing practical systems that can assess traffic conditions at overcrowded grade crossings. The main challenge in evaluating the congestion conditions at the crossings is the different instance classes (i.e., vehicle, train, and pedestrian) that need to be accurately detected, especially when densely packaged. In this study, a novel convolutional neural network (CNN) named dense traffic detection net (DTDNet) is developed. DTDNet proposes to integrate the Transformer Attention (TA) module for better modeling of global context information and the learning-to-match detection head for optimizing object detection and localization using a likelihood probability fashion. To train and test DTDNet, a unique grade crossing traffic image dataset including congested and normal traffic during both daytime and nighttime is established. Experimental results on the dataset show that the proposed DTDNet achieves the maximum mean average precision (mAP) value, 0.832, outperforming the other state-of-the-art (SOTA) models. Field test results with low mean average error (MAE), mean relative error (MRE), and root mean squared error (RMSE) which are 2.200, 1.890, and 0.280, respectively suggest the proposed model has a satisfying and robust performance in the field application under different environments.”.