Semantic Mining Research Articles

Token Embeddings Augmentation benefits Parameter-Efficient Fine-Tuning under long-tailed distribution

Pre-trained vision-language models, particularly those utilizing CLIP, have advanced various visual tasks. Parameter-Efficient Fine-Tuning (PEFT) on such models is a mainstream trend for downstream tasks. Despite advancements, long-tailed distribution still hampers image recognition performance in current PEFT schemes. Therefore, this paper proposes Token Embeddings Augmentation (TEA) to tackle long-tailed learning under PEFT paradigm. Based on patch token semantic mining, TEA uncovers category-specific semantic details within patch tokens to enhance token embeddings, named Patch-based Embeddings Augmentation (PEA). Then, a Probability Gate (PG) strategy is designed to effectively enrich semantic information of tail categories using enhanced embeddings. A Token Embeddings Consistency (TEC) loss is further introduced to prioritize category semantic information within tokens. Extensive experiments on multiple long-tailed distribution datasets show that our method improves the performance of various PEFT methods with different classification loss functions, especially for tail categories. Our optimal approach achieves the state-of-the-art results on multiple datasets with negligible parameters or inference latency, thus enhancing the practicality of PEFT in long-tailed distributions.

Informed sampling and recommendation of cycling routes: leveraging crowd-sourced trajectories with weighted-latent Dirichlet allocation

Attractive cycling routes can effectively promote active mobility, thus reducing the twin pressures of the population boom and the greenhouse effect. However, the existing approaches for cycling route recommendation primarily concentrate on identifying the most efficient routes while ignoring the urban spatial context, which is essential to meet the user’s particular preferences. This article proposes a novel method for informed sampling and recommending cycling routes leveraging crowd-sourced trajectories with weighted-latent Dirichlet allocation (WLDA). Precisely, spatial context mapping, incorporating a weighting mechanism into LDA, latent topics mining, and cycling route recommendation based on informed sampling are introduced. We collected 1,016 cycling trajectories around Cologne, Germany, for experimental analysis. The experimental results show that the three latent topics within the trajectories, leisure, city, and green tours, are clearly presented in the line density analysis. The insightful recommendation for unfamiliar cyclists could also be actively sampled upon the WLDA model. These findings suggest that our approach could shift the route recommendation paradigm from GIS analysis to a semantic mining perspective, yielding highly interpretable results and offering novel research avenues for applying machine learning in route planning.

A Web Semantic Mining Method for Fake Cybersecurity Threat Intelligence in Open Source Communities

In order to overcome the challenges of inadequate classification accuracy in existing fake cybersecurity threat intelligence mining methods and the lack of high-quality public datasets for training classification models, we propose a novel approach that significantly advances the field. We improved the attention mechanism and designed a generative adversarial network based on the improved attention mechanism to generate fake cybersecurity threat intelligence. Additionally, we refine text tokenization techniques and design a detection model to detect fake cybersecurity threats intelligence. Using our STIX-CTIs dataset, our method achieves a remarkable accuracy of 96.1%, outperforming current text classification models. Through the utilization of our generated fake cybersecurity threat intelligence, we successfully mimic data poisoning attacks within open-source communities. When paired with our detection model, this research not only improves detection accuracy but also provides a powerful tool for enhancing the security and integrity of open-source ecosystems.

Transferable dual multi-granularity semantic excavating for partially relevant video retrieval

Partially Relevant Video Retrieval (PRVR) aims to retrieve partially relevant videos from many unlabeled and untrimmed videos according to the query, which is defined as the multiple instance learning problem. The challenge of PRVR is that it utilizes untrimmed videos, which are much closer to reality. The existing methods excavate video-text semantic consistency information insufficiently and lack the capacity to highlight the semantics of key representations. To tackle these issues, we propose a transferable dual multi-granularity semantic excavating network, called T-D3N, to focus on enhancing the learning of dual-modal representations. Specifically, we first introduce a novel transferable textual semantic learning strategy by designing Adaptive Multi-scale Semantic Mining (AMSM) component to excavate significant textual semantic from multiple perspectives. Second, T-D3N distinguishes the feature differences from the frame-wise perspective to better perform contrastive learning between positive and negative samples in the video feature domain, which can further distance the positive and negative samples and improve the probability of positive samples being retrieved by query. Finally, our model constructs multi-grained video temporal dependencies and conducts cross-grained core feature perception, which enables more sufficient multimodal interactions. Extensive experiments are performed on three benchmarks, i.e., ActivityNet Captions, Charades-STA, and TVR, our T-D3N achieves state-of-the-art results. Furthermore, we also confirm that our model is transferable on a broad range of multimodal tasks such as T2VR, VMR, and MMSum.

Deep Siamese Network for annual change detection in Beijing using Landsat satellite data

The detection of spatiotemporal changes in land use/land cover (LULC-SC) plays a paramount role in the analysis of smart cities, it can describe complex urban distribution, functions, and patterns. Chinas capital city has been developing rapidly in the past two decades, however, there are only few long-term studies on an annual scale of LULC-SC. To fill this research gap, we propose a remote sensing parallel framework for the detection of LULC-SC based on the combination of the Deep Siamese Network and long time series, which focuses on the spatial semantic information at the object level. A Landsat time series from 2002 to 2022 serves as input satellite data. First, we focus on building graph constructions at the object level and then use an autonomously constructed deeper-feature graph convolutional network (DF-GCN) to mine deeper features, spatial semantic, and relationships at the object level. Finally, the Siamese Network recognizes the changes in the spatial semantic tensors of long time series of Landsat images and quickly maps LULC-SC. The results prove that the proposed spatiotemporal change detection framework is effective in LULC-SC in Beijing. Compared with other networks, the optimal accuracy of semantic mining based on DF-GCN can reach about 90%. Over the past two decades, the LULC-SC of Beijing has changed in a complex way, with urbanization occurring primarily through the replacement of farmland. Consequently, the proposed framework can generate accurate LULC change maps at high temporal frequencies, which can contribute to a better comprehension of sustainable urban development and planning.

A hybrid deep semantic mining method considering fuzzy expressions for the automatic recognition of construction safety hazard information

Safety hazards are a key consideration in construction management. The efficient recognition of safety hazard information can help managers formulate safety hazard management measures and improve the efficiency of construction safety management. However, construction site safety hazard data are stored in semistructured and unstructured text formats, which cannot be directly converted into understandable and usable information. Moreover, safety hazard text contains many fuzzy expressions, thereby increasing the difficulty of text semantic analysis; thus, how to accurately mine safety hazard information from complex and diverse text data is an urgent problem that must be solved. In consideration of this problem, we propose a bidirectional long short-term memory (BiLSTM) method with a fuzzy word vector and self-attention mechanism (FSABiLSTM) to automatically recognize safety hazard information. This method adopts TextRank and Word2vec to calculate the fuzzy word vector and process fuzzy expressions in safety hazard text. The safety hazard text semantic features are deeply extracted based on BiLSTM and a fuzzy word vector, and the extracted semantic features are analyzed via a self-attention mechanism. Actual construction safety hazard text is used to verify the reliability and applicability of the method, and the results indicate that the accuracy of this method, which outperforms existing machine learning methods, is 91.70%. In addition, the FSABiLSTM method can be used to automatically evaluate the risk degree of safety hazards; this use is beneficial to managing and controlling safety hazards. Concerning safety hazard text data, this study provides a new deep mining approach that can enhance safety management efficiency.

Holistic-Guided Disentangled Learning With Cross-Video Semantics Mining for Concurrent First-Person and Third-Person Activity Recognition.

The popularity of wearable devices has increased the demands for the research on first-person activity recognition. However, most of the current first-person activity datasets are built based on the assumption that only the human-object interaction (HOI) activities, performed by the camera-wearer, are captured in the field of view. Since humans live in complicated scenarios, in addition to the first-person activities, it is likely that third-person activities performed by other people also appear. Analyzing and recognizing these two types of activities simultaneously occurring in a scene is important for the camera-wearer to understand the surrounding environments. To facilitate the research on concurrent first- and third-person activity recognition (CFT-AR), we first created a new activity dataset, namely PolyU concurrent first- and third-person (CFT) Daily, which exhibits distinct properties and challenges, compared with previous activity datasets. Since temporal asynchronism and appearance gap usually exist between the first- and third-person activities, it is crucial to learn robust representations from all the activity-related spatio-temporal positions. Thus, we explore both holistic scene-level and local instance-level (person-level) features to provide comprehensive and discriminative patterns for recognizing both first- and third-person activities. On the one hand, the holistic scene-level features are extracted by a 3-D convolutional neural network, which is trained to mine shared and sample-unique semantics between video pairs, via two well-designed attention-based modules and a self-knowledge distillation (SKD) strategy. On the other hand, we further leverage the extracted holistic features to guide the learning of instance-level features in a disentangled fashion, which aims to discover both spatially conspicuous patterns and temporally varied, yet critical, cues. Experimental results on the PolyU CFT Daily dataset validate that our method achieves the state-of-the-art performance.

E-commerce Webpage Recommendation Scheme Base on Semantic Mining and Neural Networks

In e-commerce websites, web mining web page recommendation technology has been widely used. However, recommendation solutions often cannot meet the actual application needs of online shopping users. To address this problem, this paper proposes an e-commerce web page recommendation solution that combines semantic web mining and BP neural networks. First, the web logs of user searches are processed and 5 features are extracted: content priority, time consumption priority, online shopping users' explicit/implicit feedback on the website, recommendation semantics and input deviation amount. Then, these features are used as input features of the BP neural network to classify and identify the priority of the final output web page. Finally, the web pages are sorted according to priority and recommended to users. This project uses book sales webpages as samples for experiments. The results show that this solution can quickly and accurately identify the webpages required by users.

Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations

Large Language Models (LLMs) have revolutionized natural language processing tasks, demonstrating their exceptional capabilities in various domains. However, their potential for graph semantic mining in job recommendations remains largely unexplored. This paper focuses on unveiling the capability of large language models in understanding behavior graphs and leveraging this understanding to enhance recommendations in online recruitment, including promoting out-of-distribution (OOD) applications. We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs and uncover underlying patterns and relationships. Specifically, we propose a meta-path prompt constructor that aids LLM recommender in grasping the semantics of behavior graphs for the first time and design a corresponding path augmentation module to alleviate the prompt bias introduced by path-based sequence input. By facilitating this capability, our framework enables personalized and accurate job recommendations for individual users. We evaluate the effectiveness of our approach on comprehensive real-world datasets and demonstrate its ability to improve the relevance and quality of recommended results. This research not only sheds light on the untapped potential of large language models but also provides valuable insights for developing advanced recommendation systems in the recruitment market. The findings contribute to the growing field of natural language processing and offer practical implications for enhancing job search experiences.

FTUNet: A Feature-Enhanced Network for Medical Image Segmentation Based on the Combination of U-Shaped Network and Vision Transformer

Semantic Segmentation has been widely used in a variety of clinical images, which greatly assists medical diagnosis and other work. To address the challenge of reduced semantic inference accuracy caused by feature weakening, a pioneering network called FTUNet (Feature-enhanced Transformer UNet) was introduced, leveraging the classical Encoder-Decoder architecture. Firstly, a dual-branch Encoder is proposed based on the U-shaped structure. In addition to employing convolution for feature extraction, a Layer Transformer structure (LTrans) is established to capture long-range dependencies and global context information. Then, an Inception structural module focusing on local features is proposed at the Bottleneck, which adopts the dilated convolution to amplify the receptive field to achieve deeper semantic mining based on the comprehensive information brought by the dual Encoder. Finally, in order to amplify feature differences, a lightweight attention mechanism of feature polarization is proposed at Skip Connection, which can strengthen or suppress feature channels by reallocating weights. The experiment is conducted on 3 different medical datasets. A comprehensive and detailed comparison was conducted with 6 non-U-shaped models, 5 U-shaped models, and 3 Transformer models in 8 categories of indicators. Meanwhile, 9 kinds of layer-by-layer ablation and 4 kinds of other embedding attempts are implemented to demonstrate the optimal structure of the current FTUNet.

An Adaptive Semantic Mining Framework for Heterogeneous Information Network Embedding

Heterogeneous information network (HIN) embedding aims to map heterogeneous nodes to the low-dimensional vector space. The existing embedding models cannot determine the optimal length of semantics automatically and reveal full semantic information adaptively for different heterogeneous networks. To address this challenge, an HIN embedding model with adaptive semantic mining is proposed. First, we project heterogeneous nodes into the same space and aggregate the features of target types in the first-degree range. Then, the semantics of different node types is combined through the attention mechanism, and latent meta-paths are mined using the attention coefficients. Finally, multiple feature aggregation layers are stacked with residual blocks. The residual weights control the proportion of semantics transferred between layers to aggregate more distal features selectively. In addition, we designed the Selected DropLink unit to remove links which transfer negative information, which can further improve the resistance of model to over-smoothing. Experiments show that our model can obtain more accurate embedding results and can automatically mine complex semantic connections between heterogeneous nodes without prior definition of meta-path and semantic depth.

The role of social media data analytics in rural tourism market trend forecasting

Abstract The rapid development of social media provides new means for market trend and quotation prediction. This paper realizes market trend prediction by performing TF-IDF keyword extraction and vectorization on social media data, improving the deep typical correlation analysis to realize semantic mining, and constructing a consumer intention mining method based on social media data. The rural tourism market trend prediction for Haikou City, Hainan Province, China focuses on examining the number of tourists and the economic income of rural tourism. The future number of inbound tourists and the total number of tourists in Haikou City will still show an upward trend overall, with the total number of tourists predicted to reach 73,641,100 in 2028. With the continuous development of rural tourism, the tourism economy of Haikou City reaches 67.068 billion in 2022, an increase of 259.67% from 2008, and is predicted to double in 2027, and is expected to reach 148.282 billion in 2028. The number of overseas tourists and foreign exchange earnings from rural tourism will continue to increase.

MRAN: Multimodal relationship-aware attention network for fake news detection

Existing multimodal fake news detection methods face challenges in jointly capturing the intramodality and cross-modal correlation relationships between image regions and text fragments. Additionally, these methods lack comprehensive hierarchical semantics mining for text. These limitations result in ineffective utilization of multimodal information and impact detection performance. To address these issues, we propose a multimodal relationship-aware attention network (MRAN), which consists of three main steps. First, a multi-level encoding network is employed to extract hierarchical semantic feature representations of text, while the visual feature extractor VGG19 learns image feature representations. Second, the captured text and image representations are input into the relationship-aware attention network, which generates high-order fusion features by calculating the similarity between information segments within modalities and cross-modal similarity. Finally, the fusion features are passed through a fake news detector, which identifies fake news. Experimental results on three benchmark datasets demonstrate the effectiveness of MRAN, highlighting its strong detection performance.

Sequential data-driven carbon peaking path simulation research of the Yangtze River Delta urban agglomeration based on semantic mining and heuristic algorithm optimization

In this paper, 41 prefecture level cities in the Yangtze River Delta (YRD) are selected as the research objects. Based on the semantic mining model, the policy data information from three dimensions namely national, provincial and municipal levels is integrated to build data-driven multiple scenarios. Then the long short-term memory neural network optimized by particle swarm optimization (PSO-LSTM) is further used to predict the peaking time of the YRD urban agglomeration under multiple scenarios. The results show that the YRD urban agglomeration as a whole can achieve carbon peaking on time under all three simulation paths, but the peaking situation of specific cities within the region will be significantly different due to differentiated policy orientations. Under the business as usual scenario, some cities in the north of the region cannot achieve carbon peaking by 2030. Under the stable growth scenario, more cities cannot achieve carbon peaking on time. However, under the low carbon scenario, all cities within the region can achieve carbon peaking by 2030. Therefore, when exploring effective carbon peaking paths, adopting more stringent policies and measures to limit carbon emissions can help achieve the regional carbon peaking goal earlier, ensure that all cities within the region can achieve carbon peaking on time, and can also narrow the gap between cities within the region in terms of peaking time. This paper explores a new scenario analysis parameter setting and carbon emission prediction method, which overcomes the subjectivity of current scenario analysis research in parameter setting to a certain extent, and also provides a scientific reference for predicting the diversified development trend of carbon emission.

Comprehensive mining of information in Weakly Supervised Semantic Segmentation: Saliency semantics and edge semantics

In the studies of Weakly Supervised Semantic Segmentation (WSSS) with image-level labels, there is an issue of incomplete semantic information, which we summarize as insufficient saliency semantic mining and neglected edge semantics. We proposes a two-stage framework, Saliency Semantic Full Mining-Edge Semantic Mining (SSFM-ESM), which views WSSS from the perspective of comprehensive information mining. In the first stage, we rely on SSFM to address the insufficient saliency semantic mining. The network learns feature representations consistent with salient regions via the proposed pixel-level class-agnostic distance loss. Then, the full saliency semantic information is mined by explicitly receiving pixel-level feedback. The initial pseudo-label with complete saliency semantic information can be obtained after the first stage. In the second stage, we focus on the mining of edge semantic information through the proposed edge semantic mining module. Specifically, we guide the initial pseudo-label avoid learning about false semantic information and obtain high-confidence edge semantics. The self-correction ability of the segmentation network is also fully utilized to obtain more edge semantic information. Extensive experiments are conducted on the PASCAL VOC 2012 and MS COCO 2014 datasets to verify the feasibility and superiority of this approach.

A Self-Attention Model for Next Location Prediction Based on Semantic Mining

With the rise in the Internet of Things (IOT), mobile devices and Location-Based Social Network (LBSN), abundant trajectory data have made research on location prediction more popular. The check-in data shared through LBSN hide information related to life patterns, and obtaining this information is helpful for location prediction. However, the trajectory data recorded by mobile devices are different from check-in data that have semantic information. In order to obtain the user’s semantic, relevant studies match the stay point to the nearest Point of Interest (POI), but location error may lead to wrong semantic matching. Therefore, we propose a Self-Attention model for next location prediction based on semantic mining to predict the next location. When calculating the semantic feature of a stay point, the first step is to search for the k-nearest POI, and then use the reciprocal of the distance from the stay point to the k-nearest POI and the number of categories as weights. Finally, we use the probability to express the semantic without losing other important semantic information. Furthermore, this research, combined with sequential pattern mining, can result in richer semantic features. In order to better perceive the trajectory, temporal features learn the periodicity of time series by the sine function. In terms of location features, we build a directed weighted graph and regard the frequency of users visiting locations as the weight, so the location features are rich in contextual information. We then adopt the Self-Attention model to capture long-term dependencies in long trajectory sequences. Experiments in Geolife show that the semantic matching of this study improved by 45.78% in TOP@1 compared with the closest distance search for POI. Compared with the baseline, the model proposed in this study improved by 2.5% in TOP@1.

Addressing challenges in multi-label text classification: A joint attention and shared semantic space approach

The task of multi-label text classification involves assigning a set of related labels to a given document. However, there are three main problems with this task. Firstly, the joint modeling of label-text and label-label relationships is inadequate. Secondly, the semantic mining of the label itself is insufficient. Lastly, the utilization of the internal structure information of the label is ignored. To address these issues, a new multi-label text classification method has been proposed. This method is based on joint attention and shared semantic space. The joint multi-head attention mechanism models the relationship between labels and documents as well as the relationship between labels simultaneously. This helps to avoid error transmission and utilizes the interaction information between them. The decouple shared semantic space embedding method improves the method of using labels semantic information and reduces deviation in the phase of modeling correlation. The hierarchical hinting method based on prior knowledge relies on the prior knowledge in the pre-trained model to exploit the labels hierarchy information. Experimental results have shown that this proposed method is superior to existing multi-label text classification methods in public datasets.

Generating Chinese Event Extraction Method Based on ChatGPT and Prompt Learning

Regarding the scarcity of annotated data for existing event extraction tasks and the insufficient semantic mining of event extraction models in the Chinese domain, this paper proposes a generative joint event extraction model to improve existing models in two aspects. Firstly, it utilizes the content generation capability of ChatGPT to generate annotated data corpora for event extraction tasks and trains the model using supervised learning methods adapted to downstream tasks. Secondly, explicit entity markers and event knowledge are added to the text to construct generative input templates, enhancing the performance of event extraction. To validate the performance of this model, experiments are conducted on DuEE1.0 and Title2Event public datasets, and the results show that both data enhancement and prompt learning based on ChatGPT effectively improve the performance of the event extraction model, and the F1 values of the events extracted by the CPEE model proposed in this paper reach 85.1% and 59.9% on the two datasets, respectively, which are comparable to the existing models’ values of 1.3% and 10%, respectively; moreover, on the Title2Event dataset, the performance of different models on the event extraction task can be gradually improved as the data size of the annotated corpus of event extraction generated using ChatGPT increases.

Evolutionary dynamics of the city brand influence of top ten global cities: Characteristics analysis driven by global news big data and intelligent semantic mining

This study focuses on the long-term comparison of the city brand influence constructed and reflected by global news media. Specifically, from the analytical perspective of two paths and five characteristics, this study empirically and quantitatively assesses and deconstructs the evolutionary dynamics of the city brand influence of the top ten Globalization and the World Research Group (GaWC) Alpha+ or above global cities over five consecutive years based on intelligent semantic embedding mining on the Global Database of Events, Language, and Tone (GDELT) full-sample global news big data, along with statistical analysis and machine learning. The results indicate that the top ten global cities' city brand influence over five consecutive years fluctuated from high to low and then rebounded. The hierarchical tiers of the Alpha++ and Alpha+ global cities' city brand influence were relatively stable. However, there are inner tensions in the evolutionary trend, such as the uneven rank differences between the city brand influence soft power and the hard power of these cities, and the intersection of clustering categories between the Alpha++ and Alpha+ global cities. The findings and data of this study provide concrete references and guidelines for policymaking toward future competition of top global cities' city brand influence.

Encrypted DNS Traffic Analysis for Service Intention Inferring

Service intention refers to what service or which service the server provides. The former includes service classification, service type, or service behavior classification. The latter contains service content classification, such as shopping online or uploading and downloading, etc.. Port-based classification and payload-based classification are two widely used service classification schemes, both of which have many limitations, such as only focusing on server-side scenarios or just designing for non-encrypted requests. In this paper, we propose an encryption-independent approach from a network-side perspective by analyzing the communication behavior of the IPs. Firstly, we identify similar service behavior clusters by employing service influence metrics. Then, we devise a semantic mining mechanism to infer whether they serve a fixed user group or provide interactive service. Finally, we use open-source benchmark datasets, synthetic datasets, and the real Netflow data collected from the China Education Research Network backbone (CERNET) to verify our proposal. Experimental results demonstrate that the accuracy and recall rate of the proposed approach is better than other similar state-of-the-art methods. Besides, our work can also distinguish malicious behavior clusters. Extensive experiments demonstrate that our work is efficient for network management and security monitoring.