Local Context Features Research Articles

The named entity recognition of vessel power equipment fault using the multi-details embedding model

The fault diagnosis of vessel power equipment is established by the manual work with low efficiency. The knowledge graph(KG) usually is applied to extract the experience and operation logic of controllers into knowledge, which can enrich the means of fault judgment and recovery decision. As an important part of KG building, the performance of named entity recognition (NER) is critical to the following tasks. Due to the challenges of information insufficiency and polysemous words in the entities of vessel power equipment fault, this study adopts the fusion model of Bidirectional Encoder Representations from Transformers (BERT), revised Convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM), and conditional random field (CRF). Firstly, the adjusted BERT and revised CNN are respectively adopted to acquire the multiple embeddings including semantic information and contextual glyph features. Secondly, the local context features are effectively extracted by adopting the channel-wised fusion structures. Finally, BiLSTM and CRF are respectively adopted to obtain the semantic information of the long sequences and the prediction sequence labels. The experimental results show that the performance of NER by the proposed model outperforms other mainstream models. Furthermore, this work provides the foundation of the tasks of intelligent diagnosis and NER in other fields.

A representation learning framework for stock movement prediction

The learning of high-quality stock representations is one of the keys to predicting stock movements effectively. Current studies have been negatively impacted by stochasticity in stock prices, resulting in inadequate representation learnt by the models. We present an end-to-end stock movement prediction framework (CLSR) utilizing contrastive learning to exploit the correlation between intra-day data and enhance stock representation in order to improve the accuracy of stock movement prediction. In addition, a hybrid encoding network is developed to extract long-range dependencies and local contextual features in stock data, making the feature representation more complete. To further improve the prediction accuracy of the model, historical state information is added to the intra-day stock data. Our experiments on CSI-500 show that the proposed method outperforms state-of-the-art solutions. The proposed method is also validated by analyzing the representation space thoroughly.

Uniting Multi-Scale Local Feature Awareness and the Self-Attention Mechanism for Named Entity Recognition

In recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although the mainstream methods for Chinese medical named entity recognition can effectively capture global context information, they ignore the potential local information in sentences, and hence cannot extract the local context features through an efficient framework. To overcome these challenges, this paper constructs a diabetes corpus and proposes the RMBC (RoBERTa Multi-scale CNN BiGRU Self-attention CRF) model. This model is a named entity recognition model that unites multi-scale local feature awareness and the self-attention mechanism. This paper first utilizes RoBERTa-wwm to encode the characters; then, it designs a local context-wise module, which captures the context information containing locally important features by fusing multi-window attention with residual convolution at the multi-scale and adds a self-attention mechanism to address the restriction of the bidirectional gated recurrent unit (BiGRU) capturing long-distance dependencies and to obtain global semantic information. Finally, conditional random fields (CRF) are relied on to learn of the dependency between adjacent tags and to obtain the optimal tag sequence. The experimental results on our constructed private dataset, termed DNER, along with two benchmark datasets, demonstrate the effectiveness of the model in this paper.

Uncertainty-guided joint attention and contextual relation network for person re-identification

Due to the influence of factors such as camera angle and pose changes, some salient local features are often suppressed in person re-identification tasks. Moreover, many existing person re-identification methods do not consider the relation between features. To address these issues, this paper proposes two novel approaches: (1) To solve the problem of being confused and misidentified when local features of different individuals have similar attributes, we design a contextual relation network that focuses on establishing the relationship between local features and contextual features, so that all local features of the same person both contain contextual information. (2) To fully and correctly express key local features, we propose an uncertainty-guided joint attention module. The module focuses on the joint representation of individual pixels and local spatial features to enhance the credibility of local features. Finally, our method achieves competitive performance on four widely recognized datasets compared with state-of-the-art methods.

MAF-Net: A multi-scale attention fusion network for automatic surgical instrument segmentation

Accurate localization of surgical instruments is of utmost importance for precise robot-assisted surgeries. With the development of artificial intelligence, deep convolutional neural networks (DCNNs) have been widely employed for automatic image segmentation, owing to their strong ability to generate contextual features, especially in the encoder–decoder framework. However, existing segmentation networks lack the feature capturing capability on micro objects and have shortcomings in processing local semantic features. These limitations can affect the precise segmentation of surgical instruments. In response to these issues, this paper proposes a multi-scale attention fusion network called MAF-Net, which comprises residual dense module, a multi-scale atrous convolution (MSAC) module, and an attention fusion module (AFM). To improve the processing ability of local contextual features, we propose replacing skip connections with residual dense module to acquire stronger contexts. Furthermore, a MSAC module is proposed for local feature enhancement, thereby enhancing attention on multi-scale features. In addition, an AFM module is introduced to integrate multi-scale information by cross-scale feature fusion. Experimental results, using two public datasets, Endovis2017 and kvasir-instrument, demonstrate that the proposed network has the ability to achieve precise surgical instrument segmentation and outperforms related advanced methods.

Learning local contextual features for 3D point clouds semantic segmentation by attentive kernel convolution

Learning local contextual features for 3D point clouds semantic segmentation by attentive kernel convolution

An Anchor-Free Network for Increasing Attention to Small Objects in High Resolution Remote Sensing Images

Aimed at the problems of small object detection in high resolution remote sensing images, such as difficult detection, diverse scales, and dense distribution, this study proposes a new method, DCE_YOLOX, which is more focused on small objects. The method uses depthwise separable deconvolution for upsampling, which can effectively recover lost feature information and combines dilated convolution and CoTNet to extract local contextual features, which can make full use of the hidden semantic information. At the same time, EcaNet is added to the enhanced feature extraction network of the baseline model to make the model more focused on information-rich features; secondly, the network input resolution is optimized, which can avoid the impact of image scaling to a certain extent and improve the accuracy of small object detection. Finally, CSL is used to calculate the angular loss to achieve the rotated object detection of remote sensing images. The proposed method in this study achieves 83.9% accuracy and 76.7% accuracy for horizontal object detection and rotationally invariant object detection, respectively, in the DOTA remote sensing dataset; it even achieves 96% accuracy for rotationally invariant object detection in the HRSC2016 dataset. It can be concluded that our algorithm has a better focus on small objects, while it has an equally good focus on other objects and is well suited for applications in remote sensing, and it has certain reference significance for realizing the detection of small objects in remote sensing images.

Double-branch feature fusion transformer for hyperspectral image classification

Deep learning methods, particularly Convolutional Neural Network (CNN), have been widely used in hyperspectral image (HSI) classification. CNN can achieve outstanding performance in the field of HSI classification due to its advantages of fully extracting local contextual features of HSI. However, CNN is not good at learning the long-distance dependency relation and dealing with the sequence properties of HSI. Thus, it is difficult to continuously improve the performance of CNN-based models because they cannot take full advantage of the rich and continuous spectral information of HSI. This paper proposes a new Double-Branch Feature Fusion Transformer model for HSI classification. We introduce Transformer into the process of HSI on account of HSI with sequence characteristics. The two branches of the model extract the global spectral features and global spatial features of HSI respectively, and fuse both spectral and spatial features through a feature fusion layer. Furthermore, we design two attention modules to adaptively adjust the importance of spectral bands and pixels for classification in HSI. Experiments and comparisons are carried out on four public datasets, and the results demonstrate that our model outperforms any compared CNN-Based models in terms of accuracy.

Improvement of Retinal Vessel Segmentation Method Based on U-Net

Retinal vessel segmentation remains a challenging task because the morphology of the retinal vessels reflects the health of a person, which is essential for clinical diagnosis. Therefore, achieving accurate segmentation of the retinal vessel shape can determine the patient’s physical condition in a timely manner and can prevent blindness in patients. Since the traditional retinal vascular segmentation method is manually operated, this can be time-consuming and laborious. With the development of convolutional neural networks, U-shaped networks (U-Nets) and variants show good performance in image segmentation. However, U-Net is prone to feature loss due to the operation of the encoder convolution layer and also causes the problem of mismatch in the processing of contextual information features caused by the skip connection part. Therefore, we propose an improvement of the retinal vessel segmentation method based on U-Net to segment retinal vessels accurately. In order to extract more features from encoder features, we replace the convolutional layer with ResNest network structure in feature extraction, which aims to enhance image feature extraction. In addition, a Depthwise FCA Block (DFB) module is proposed to deal with the mismatched processing of local contextual features by skip connections. Combined with the two public datasets on retinal vessel segmentation, namely DRIVE and CHASE_DB1, and comparing our method with a larger number of networks, the experimental results confirmed the effectiveness of the proposed method. Our method is better than most segmentation networks, demonstrating the method’s significant clinical value.

Multi-scale local feature enhanced transformer network for pavement crack detection

目的 道路裂缝是路面病害的早期征兆。定期监测路面状况、及时准确地发现路面裂缝对于交通养护机构降低成本、保证路面结构的可靠性和耐久性以及提高驾驶安全性、舒适性有重要意义。目前基于卷积神经网络的深度学习模型在长距离依赖建模方面存在不足，模型精度难以满足真实路面环境下的裂缝检测任务。一些模型通过引入空间/通道注意力机制进行长距离依赖建模，但是会导致计算量和计算复杂程度增加，无法实现实时检测。鉴于此，本文提出一种基于Transformer编码—解码结构的深度神经网络道路裂缝检测模型CTNet （crack transformernetwork）。方法 该模型主要由Transformer注意力模块、多尺度局部特征增强模块、上采样模块和跨越连接4部分构成。采用Transformer注意力机制能更有效提取全局和长距离依赖关系，克服传统卷积神经网络表征输入信息的短距离相关缺陷。同时，为适应裂缝尺寸变化多样性，将Transformer与多尺度局部特征增强模块相结合，从而有效整合不同尺度局部信息，克服Transformer局部特征建模不足。结果 通过与DeepCrack模型在不同裂缝检测数据集中的比较表明，本文提出的多尺度局部特征增强Transformer网络能快速、准确地分割路面裂缝，且效率更优。定量研究结果表明，CTNet在更有挑战性的CrackLS315数据集中的精度、召回率和F1值达到91.38%、80.38%和85.53%，明显优于对比方法。在CrackWH100数据集中，精度、召回率和F1值进一步提升，分别达到92.70%、90.52%和91.60%。此外，CTNet的训练速率提升至DeepCrack模型的6.78倍。结论 CTNet可以实现强噪声背景下的道路裂缝检测，检测效果优于目前最优方法，且参数量小，易于训练和部署。;Objective The pavement-relevant inspection is focused on the optimization for pavement cracks early-alarming detection and the preservation of pavement structure. However，conventional image processing-based techniques are laborintensive and time-consuming，such as edge detection，threshold segmentation，template matching，and morphology operations. It is challenged to the geometric and spectral complexities of pavement crack and its contexts（e. g. ，illumination variation，oil or water stains，and shadows caused by trees and vehicles）. The convolution neural network（CNN）based deep learning image processing techniques have been developing intensively. However，the CNN-based methods are less effective in long-range dependency modeling，which may cause insufficient detection results in complicated road surface scenarios. Some works are related to attention mechanisms like spatial or channel attention modules，and self-attention modules. However，these attention mechanism-based operations are still challenged for their sophistication and computational cost. Method To detect pavement cracks efficiently and effectively，we develop a novel Transformer-based encoderdecoder neural network，called CTNet，which consists of Transformer blocks，multi-scale local feature enhanced blocks， upsampling blocks，and skip connections. The CTNet can achieve more long-range dependency and global receptive field in terms of multi-head self-attention-based Transformer mechanism. Although Transformer is featured by high running efficiency and low computational overhead demand，it is infeasible to model local contextual information because Token generation can break the connections of neighboring regions. Thus，to capture more multi-scale local information，we design a multi-scale local feature-enhanced block in terms of a multiple dilation ratios-relevant dilation convolution block. Especially，the designed multi-scale local feature enhancement block is melted into each Transformer block for local information complement. Both of local and global low-level contextual features can be captured for feature enhancement. Afterwards，a novel decoder path is implemented to extract high-level features. The decoder consists of the Transformer blocks similar to the up-sampling blocks and the spatial details can be restored for end-to-end segmentation. Result To demonstrate the efficiency and effectiveness of our proposed CTNet，a series of comparative analyses and ablation studies are carried out on three datasets. First，the CTNet can optimize running efficiency，as well as comparable computation overhead and complexity compared to the current UNet，SegNet，DeepCrack，and SwinUnet. Second，CTNet is 6. 78 times faster than the second-best DeepCrack model in terms of training speed. On CrackLS315 dataset，quantitative analyses are also showed that the optimal CTNet is obtained a precision of 91. 38%，a recall of 80. 38%，and a F1 measure of 85. 53% of each；on CrackWH100 dataset，CTNet can obtain a precision of 92. 70%，a recall of 90. 52%，and a F1 measure 91. 60% of each as well. However，it is still challenged to lack of local information when pure-Transformer-based Swin-UNet performed not well compared to fully convolution networks. Furthermore，the CTNet is insufficient to converge when the local blocksenhanced are removed. In summary，the Transformer-based CTNet is beneficial to multi-scenario pavement cracks in terms of the global receptive field. The CTNet can get pavement crack detection results consistently. Conclusion The proposed CTNet has its potentials to deal with noisy pavement images for pavement crack detection.

Global–Local 3-D Convolutional Transformer Network for Hyperspectral Image Classification

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to learn complex characteristics of long-range sequences. Meanwhile, vision transformer allows us to learn long-range dependencies in a global view, but local region features are ignored. To overcome these limitations, we propose a novel method entitled global-local three-dimensional convolutional transformer network (GTCT), where 3-D convolution is embedded in a dual-branch transformer to simultaneously capture global-local associations in both spectral and spatial domains. In particular, the global-local spectral convolutional transformer (GECT) is designed to exploit global spectral sequence signatures and local spectral relationships between bands. Symmetrically, the global-local spatial convolutional transformer (GACT) is devised to exploit local spatial context features and global interactions among different pixels. In addition, multiscale global-local spectral-spatial information is adaptively fused with trainable weights by the weighted multiscale spectral-spatial feature interaction (WMSFI) module. It is worth noting that a spectral-spatial global attention mechanism (SSGAM) is incorporated into multi-head convolutional attention to further integrate discriminative spectral-spatial information. Extensive experiments on four HSI datasets, including GF-5 and ZY1-02D satellite hyperspectral images, demonstrate the superiority of the proposed GTCT method over other state-of-the-art algorithms with fewer parameters and lower floating-point operations (FLOPs) in practical applications.

Adaptive Local Context and Syntactic Feature Modeling for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis is a fine-grained sentiment analysis task that consists of two types of subtasks: aspect term extraction and aspect sentiment classification. In the aspect term extraction task, current methods suffer from the lack of fine-grained information in aspect term extraction and difficulty in identifying aspect term boundaries. In the aspect sentiment classification task, the current aspect sentiment classifier cannot adapt itself to the text and determine the local context. To address these two challenges, this work proposes an adaptive semantic relative distance approach based on dependent syntactic analysis, which uses adaptive semantic relative distance to determine the appropriate local context for each text and increase the accuracy of sentiment analysis. Meanwhile, the study also predicts the current word labels by combining local information features extracted by local convolutional neural networks and global information features to precisely locate the word labels. In two subtasks, our proposed model improves accuracy and F1 scores on the SemEval-2014 Task 4 Restaurant and Laptop datasets compared to the state-to-the-art approaches, especially in the aspect sentiment classification subtask.

VANet: a medical image fusion model based on attention mechanism to assist disease diagnosis

BackgroundToday’s biomedical imaging technology has been able to present the morphological structure or functional metabolic information of organisms at different scale levels, such as organ, tissue, cell, molecule and gene. However, different imaging modes have different application scope, advantages and disadvantages. In order to improve the role of medical image in disease diagnosis, the fusion of biomedical image information at different imaging modes and scales has become an important research direction in medical image. Traditional medical image fusion methods are all designed to measure the activity level and fusion rules. They are lack of mining the context features of different modes of image, which leads to the obstruction of improving the quality of fused images.MethodIn this paper, an attention-multiscale network medical image fusion model based on contextual features is proposed. The model selects five backbone modules in the VGG-16 network to build encoders to obtain the contextual features of medical images. It builds the attention mechanism branch to complete the fusion of global contextual features and designs the residual multiscale detail processing branch to complete the fusion of local contextual features. Finally, it completes the cascade reconstruction of features by the decoder to obtain the fused image.ResultsTen sets of images related to five diseases are selected from the AANLIB database to validate the VANet model. Structural images are derived from MR images with high resolution and functional images are derived from SPECT and PET images that are good at describing organ blood flow levels and tissue metabolism. Fusion experiments are performed on twelve fusion algorithms including the VANet model. The model selects eight metrics from different aspects to build a fusion quality evaluation system to complete the performance evaluation of the fused images. Friedman’s test and the post-hoc Nemenyi test are introduced to conduct professional statistical tests to demonstrate the superiority of VANet model.ConclusionsThe VANet model completely captures and fuses the texture details and color information of the source images. From the fusion results, the metabolism and structural information of the model are well expressed and there is no interference of color information on the structure and texture; in terms of the objective evaluation system, the metric value of the VANet model is generally higher than that of other methods.; in terms of efficiency, the time consumption of the model is acceptable; in terms of scalability, the model is not affected by the input order of source images and can be extended to tri-modal fusion.

TARNet: An Efficient and Lightweight Trajectory-Based Air-Writing Recognition Model Using a CNN and LSTM Network

Air-writing is a growing research topic in the field of gesture-based writing systems. This research proposes a unified, lightweight, and general-purpose deep learning algorithm for a trajectory-based air-writing recognition network (TARNet). We combine a convolutional neural network (CNN) with a long short-term memory (LSTM) network. The architecture and applications of CNN and LSTM networks differ. LSTM is good for time series prediction yet time-consuming; on the other hand, CNN is superior in feature generation but comparatively faster. In this network, the CNN and LSTM serve as a feature generator and a recognizer, optimizing the time and accuracy, respectively. The TARNet utilizes 1-dimensional separable convolution in the first part to obtain local contextual features from low-level data (trajectories). The second part employs the recurrent algorithm to acquire the dependency of high-level output. Four publicly available air-writing digit (RealSense trajectory digit), character (RealSense trajectory character), smart-band, and Abas datasets were employed to verify the accuracy. Both the normalized and nonnormalized conditions were considered. The use of normalized data required longer training times but provided better accuracy. The test time was the same as those for nonnormalized data. The accuracy for RTD, RTC, smart-band, and Abas datasets were 99.63%, 98.74%, 95.62%, and 99.92%, respectively.

Sentiment Classification of Educational Tourism Reviews Based on Parallel CNN and LSTM with Attention Mechanism

With the rapid development of the Internet and tourism, the Internet has been widely used in the tourism industry. Tourism enterprises and tourists use the Internet to publish and obtain travel-related information. Educational tourism is a new type of tourism activity. As a combination of “tourism + education,” it has gradually attracted the attention of tourists. With its convenience, fast speed, and low barrier, tourism text data provide great convenience for tourists’ sentiment calculation and have become one of the main sources of big data for tourism. However, the reviews of educational tourism have a lot of redundant information and complex sentence patterns, leading to a relatively low classification accuracy of the existing sentiment analysis algorithms. In order to effectively obtain the implicit semantic information of short text reviews for sentiment orientation recognition, a sentiment classification model for educational tourism online reviews based on parallel CNN and LSTM with multichannel attention mechanism is proposed. Firstly, Word2Vec technique is used, and based on noise word filtering, the feature words of educational tourism reviews are extracted to preprocess the input data set. Then, parallel CNN and LSTM are used to extract text local information and contextual features, and a multichannel attention mechanism is used to extract the attention values from the LSTM output. Finally, the output information of the multichannel attention mechanism is fused to effectively extract text features and focus on important words. The experimental results show that compared with other advanced methods, the proposed algorithm achieves improvements in terms of precision, recall, and F1 value and improves the AUC performance. It will help the educational tourism bases to carry out targeted development and construction in response to tourists feedback, enhance the sense of gain and happiness of tourists in educational tourism activities, improve tourism experience, and promote the rapid development of high-quality educational tourism.

A Novel Deep Learning Framework for Automatic Recognition of Thyroid Gland and Tissues of Neck in Ultrasound Image

Recognition of thyroid glands and tissues of the neck is vital for screening related diseases in ultrasound videos. This task is subjective, challenging, and dependent on the experience of sonographer in current clinical practice. The purpose is to develop a fully automated thyroid gland and tissues of neck recognition framework to assist doctors in distinguishing the boundaries of different tissues. In this paper, we propose a novel deep learning framework that consists of a feature extraction network, region proposal network, object detection head, and spatial pyramid RoIAlign-based segmentation head. Designed spatial pyramid RoIAlign can efficiently capture local and global context features, and aggregates the multiple context information that makes the result much more reliable. A large dataset is constructed to train the proposed method. The performance is evaluated using the COCO metrics. The experimental results demonstrate that the proposed deep learning method can effectively realize the automatic recognition of the thyroid gland and tissues of neck in ultrasound videos. Considering the clinical practical application scenarios, we developed an automatic recognition system of thyroid and neck tissue based on edge computing, which can expediently assist doctors in distinguishing the boundaries between different tissues.

ResDO-UNet: A deep residual network for accurate retinal vessel segmentation from fundus images

For the clinical diagnosis, it is essential to obtain accurate morphology data of retinal blood vessels from patients, and the morphology of retinal blood vessels can well help doctors to judge the patient’s condition and give targeted therapeutic measures. Conventional manual retinal blood vessel segmentation by the doctors from the fundus images is time-consuming and laborious, while it also requires the rich doctor’s expertise. With the strong context feature expression ability of deep convolutional neural networks (DCNN), it has shown a promising performance on retinal blood vessel segmentation, specially U-shape network (U-Net) and its variant. However, due to the information loss issue caused by multiple pooling operations and insufficient process issue of local context features by skip connections, most of segmentation methods still exist a certain shortcoming on accurate fine vessel detection. To address this issue, based on the encoder–decoder framework, a novel retinal vessel segmentation network, called ResDO-UNet, is proposed to provide an automatic and end-to-end detection scheme from fundus images. To enhance feature extraction capabilities, combined with depth-wise over-parameterized convolutional layer (DO-conv), a residual DO-conv (ResDO-conv) network is proposed to act as the backbone network to acquire strong context features. In addition, to reduce the effect of information loss caused by multiple pooling operations, taking advantages of max pooling and average pooling layers, a pooling fusion block (PFB) is proposed to realize nonlinear fusion pooling. Meanwhile, faced with insufficient process of local context features by skip connections, an attention fusion block (AFB) is proposed to realize effective multi-scale feature expression. Combined with the three public available data sets on retinal vessel segmentation, including DRIVE, STARE and CHASE_DB1, the proposed segmentation network could reach a state-of-the-art detection performance compared to other related advanced work.

Multi-level context-driven interaction modeling for human future trajectory prediction

Human trajectory prediction is a challenging task with important applications such as intelligent surveillance and autonomous driving. We recognize that pedestrians in close and distant neighborhoods have different impacts on the person’s decision of future movements. Local scene context and global scene layout also affect the movement decision differently. Existing methods have not adequately addressed these interactions between humans and the multi-level contexts occurring at different spatial and temporal scales. To this end, we propose a multi-level context-driven interaction modeling (MCDIM) method for human future trajectory learning and prediction. Specifically, we construct a multilayer graph attention network (GAT) to model the hierarchical human–human interactions. An extra set of long short-term memory networks is designed to capture the correlations of these human–human interactions at different temporal scales. To model the human–scene interactions, we explicitly extract and encode the global scene layout features and local context features in the neighborhood of the person at each time step and capture the spatial–temporal information of the interactions between human and the local scene contexts. The human–human and human–scene interactions are incorporated into the multi-level GAT-based network for accurate prediction of future trajectories. We have evaluated the method on benchmark datasets: the walking pedestrians dataset provided by ETH Zurich (ETH) and the crowd data provided by the University of Cyprus. The results demonstrate that our MCDIM method outperforms existing methods, being able to generate more accurate and plausible trajectories for pedestrians. The average performance gain is 2 and 3 percentage points in terms of the average displacement error and final displacement error, respectively.

When Transformer Meets Robotic Grasping: Exploits Context for Efficient Grasp Detection

In this letter, we present a transformer-based architecture, namely TF-Grasp, for robotic grasp detection. The developed TF-Grasp framework has two elaborate designs making it well suitable for visual grasping tasks. The first key design is that we adopt the local window attention to capture local contextual information and detailed features of graspable objects. Then, we apply the cross window attention to model the long-term dependencies between distant pixels. Object knowledge, environmental configuration, and relationships between different visual entities are aggregated for subsequent grasp detection. The second key design is that we build a hierarchical encoder-decoder architecture with skip-connections, delivering shallow features from the encoder to decoder to enable a multi-scale feature fusion. Due to the powerful attention mechanism, TF-Grasp can simultaneously obtain the local information (i.e., the contours of objects), and model long-term connections such as the relationships between distinct visual concepts in clutter. Extensive computational experiments demonstrate that TF-Grasp achieves competitive results versus state-of-art grasping convolutional models and attains a higher accuracy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$97.99 \%$</tex-math></inline-formula> and 94.6% on Cornell and Jacquard grasping datasets, respectively. Real-world experiments using a 7DoF Franka Emika Panda robot also demonstrate its capability of grasping unseen objects in a variety of scenarios. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/WangShaoSUN/grasp-transformer</uri> .

Transformer-based deep learning models for the sentiment analysis of social media data

Sentiment analysis (SA) is a widely used contextual mining technique for extracting useful and subjective information from text-based data. It applies on Natural Language Processing (NLP), text analysis, biometrics, and computational linguistics to identify, analyse, and extract responses, states, or emotions from the data. The features analysis technique plays a significant role in the development and improvement of a SA model. Recently, GloVe and Word2vec embedding models have been widely used for feature extractions. However, they overlook sentimental and contextual information of the text and need a large corpus of text data for training and generating exact vectors. These techniques generate vectors for just those words that are included in their vocabulary and ignore Out of Vocabulary Words (OOV), which can lead to information loss. Another challenge for the classification of sentiments is that of the lack of readily available annotated data. Sometimes, there is a contradiction between the review and their label that may cause misclassification. The aim of this paper is to propose a generalized SA model that can handle noisy data, OOV words, sentimental and contextual loss of reviews data. In this research, an effective Bi-directional Encoder Representation from Transformers (BERT) based Convolution Bi-directional Recurrent Neural Network (CBRNN) model is proposed with for exploring the syntactic and semantic information along with the sentimental and contextual analysis of the data. Initially, the zero-shot classification is used for labelling the reviews by calculating their polarity scores. After that, a pre-trained BERT model is employed for obtaining sentence-level semantics and contextual features from that data and generate embeddings. The obtained contextual embedded vectors were then passed to the neural network, comprised of dilated convolution and Bi-LSTM. The proposed model uses dilated convolution instead of classical convolution to extract local and global contextual semantic features from the embedded data. Bi-directional Long Short-Term Memory (Bi-LSTM) is used for the entire sequencing of the sentences. The CBRNN model is evaluated across four diverse domain text datasets based on accuracy, precision, recall, f1-score and AUC values. Thus, CBRNN can be efficiently used for performing SA tasks on social media reviews, without any information loss.