Semantic Inference Research Articles

Are large language models qualified reviewers in originality evaluation?

Large language models (LLMs) are a new generation of conversational language model with impressive semantic comprehension, text generation, and knowledge inference capabilities. LLMs are significantly influencing the development of science by assisting researchers in analyzing, understanding, and grasping original knowledge in scientific papers. This study investigates LLMs’ potential as qualified reviewers in originality evaluation in zero-shot learning, utilizing a unique, manually crafted prompt. Using biomedical papers as the data source, we constructed two evaluation datasets based on Nobel Prize papers and disruptive index. The evaluation performance of multiple LLMs of different types and scales on the datasets was scrutinized through the analysis of originality score (OS), originality type (OT), and originality description (OD), all of which were generated by the LLM. Our results show that LLMs can to some extent discern papers with distinct originality level via OS; however, they appear to be overly lenient reviewers. In LLMs’ evaluation mechanism, five distinct OTs reflecting varied research contributions do not manifest independently, but together they positively influence OS. Of all the LLMs analyzed, GPT-4 stood out as able to produce the most readable ODs, effectively explaining the inference process for both OS and OT from the perspectives of completeness, logicality, and regularity.

SSA: semantic structure aware inference on CNN networks for weakly pixel-wise dense predictions without cost

SSA: semantic structure aware inference on CNN networks for weakly pixel-wise dense predictions without cost

Zero-Shot Relation Classification Through Inference on Category Attributes.

The goal of relationship classification (RC) is to predict the semantic relationship between two entities in a given sentence. With the advent of deep learning and pretrained language models, RC research has progressed by leaps and bounds. However, the current studies are focused mainly on predicting semantic relationships from a predefined set. How to recognize unseen relationships remains a challenge, which is also known as the zero-shot RC (ZSRC) task. Some ZSRC-related methods directly map relationship categories to numerical indices, constraining the model's ability to autonomously infer and understand these relationships, while others rely heavily on manual definitions. To address these issues and inspired by the way of reasoning in which humans perform RC tasks, we propose a new framework to handle the ZSRC task through inference on category attributes (ICAs). The main idea of ICA is to detect the semantic relationship between promises, which are RC sentences, and hypotheses, which are relational sentences of entities created by templates. Specifically, instead of manual design, we introduce two hypothesis templates derived from the label words (LWs) and descriptions (LDs) associated with each relationship. These templates are used to automatically convert the RC data into the textual entailment (TE) format. Furthermore, they are fine-tuned with a pretrained TE model, facilitating the acquisition of relational knowledge and enabling the generalization of semantic reasoning rules learned from seen classes to unseen classes. Moreover, to implement multirelationship semantic inference for all unseen classes, we propose an entailment difference mechanism to enhance the reasoning capability of the model. Besides the current ZSRC test setting, we also examine our method in an even more challenging setting to deal with data scarcity in real-world applications. The outstanding performance of ICA on the FewRel and Wiki-ZSL datasets demonstrates its effectiveness in the ZSRC task.

Detecting the Severity of Depression in Online Forum Data by Leveraging Implicit Semantic Inferences

Depression, a prevalent mental health disorder with global implications, exerts a profound negative influence on individuals’ lives. While the prediction of depression (as a binary classification task) is a well-established research area, depression severity detection is a new research direction with limited studies. In the context of detecting the severity of depression through online forum data, this research endeavors to offer two distinct solutions by employing Ada embeddings, GPT 3.5 Turbo, and LIWC as feature engineering techniques, while AutoSklearn serves as the ensemble learning algorithm. Notably, the outcomes of this study significantly outperform existing state-of-the-art models on both depression severity annotated datasets used in this research. The results also showcase the potential reuse nature of the proposed models in diverse data sources due to their high performance in both datasets. Furthermore, as a valuable practical outcome, a software prototype has been developed, capable of providing the depression severity level, along with associated symptoms and keywords, upon inputting an online forum post.

A benchmark dataset and semantics-guided detection network for spatial–temporal human actions in urban driving scenes

In real urban driving scenes, human actions are very complex and have the characteristic of multiple concurrent actions. It has a great significance to detect human actions in urban traffic scenes for auxiliary or autonomous driving systems. In this view, we introduce the TITAN-Human Action dataset for the task of multi-person spatial–temporal action detection in urban driving scenes. TITAN-Human Action provides the fine-grained action labels and location coordinates for 17,574 persons in the processed frames from the TITAN dataset. Furthermore, we propose a semantics-guided detection network (SGDNet) based on a semantic inference module (SIM) for spatial–temporal human action detection in urban driving scenes. SIM encodes the category labels into sentence vectors at the semantic level with prompting and embedding, utilizes graphs to represent the directed co-occurrence relations between categories, and adopts the graph convolutional network for semantic inference. SGDNet exploits the inference results of SIM to guide the visual branch in better performing human action detection, thereby achieving the integration of visual and linguistic information. We conducted experiments to evaluate SGDNet and several baseline methods on the TITAN-Human Action dataset, and reveal the generalizability of SIM in spatial–temporal human action detection. The source code and annotation files will be available at https://github.com/yyhbswyn/SGDNet.

A Defect Detection Method Based on YOLOv7 for Automated Remanufacturing

Remanufacturing of mechanical parts has recently gained much attention due to the rapid development of green technologies and sustainability. Recent efforts to automate the inspection step in the remanufacturing process using artificial intelligence are noticeable. In this step, a visual inspection of the end-of-life (EOL) parts is carried out to detect defective regions for restoration. This operation relates to the object detection process, a typical computer vision task. Many researchers have adopted well-known deep-learning models for the detection of damage. A common technique in the object detection field is transfer learning, where general object detectors are adopted for specific tasks such as metal surface defect detection. One open-sourced model, YOLOv7, is known for real-time object detection, high accuracy, and optimal scaling. In this work, an investigation into the YOLOv7 behavior on various public metal surface defect datasets, including NEU-DET, NRSD, and KolektorSDD2, is conducted. A case study validation is also included to demonstrate the model’s application in an industrial setting. The tiny variant of the YOLOv7 model showed the best performance on the NEU-DET dataset with a 73.9% mAP (mean average precision) and 103 FPS (frames per second) in inference. For the NRSD dataset, the model’s base variant resulted in 88.5% for object detection and semantic segmentation inferences. In addition, the model achieved 65% accuracy when testing on the KolektorSDD2 dataset. Further, the results are studied and compared with some of the existing defect detection models. Moreover, the segmentation performance of the model was also reported.

Application of E-Learning entertainment learning based on intelligent interactive experience in English reading assistance mode

In the process of E-Learning, interactive entertainment experiences and digital technology provide new models for strengthening the development of online education. This article studies the application of E-Learning entertainment learning based on intelligent interactive experience in English reading assistance mode. The E-Learning entertainment learning mode based on intelligent interactive experience focuses on creating interest and interaction, designing an interesting and stimulating learning environment, selecting appropriate technology tools to achieve E-Learning intelligent interactive experience, and designing appropriate learning content and activities according to learning objectives. Firstly, this article designs a semantic inference system that is more convenient in calculating and storing resources, and can perform semantic inference on uncertain knowledge. Secondly, this article delves into the basic principles of Bayesian network algorithms and explores the construction mechanism of Bayesian networks. Then compared with traditional algorithms, the accuracy and stability of this method were demonstrated. Digital technology and artificial intelligence provide new technological support for E-Learning, allowing students to better experience entertaining learning experiences and customize learning content according to personal needs, enhancing educational value. Experiments have shown that using computer-assisted English reading can effectively improve students’ English proficiency. This system not only reduces the boredom in traditional English reading, but also provides students with a brand new English learning atmosphere, which can effectively improve their learning efficiency.

Time division multiplexing based multi-spectral semantic camera for LiDAR applications

The recent progress in the development of measurement systems for autonomous recognition had a substantial impact on emerging technology in numerous fields, especially robotics and automotive applications. In particular, time-of-flight (TOF) based light detection and ranging (LiDAR) systems enable to map the surrounding environmental information over long distances and with high accuracy. The combination of advanced LiDAR with an artificial intelligence platform allows enhanced object recognition and classification, which however still suffers from limitations of inaccuracy and misidentification. Recently, multi-spectral LiDAR systems have been employed to increase the object recognition performance by additionally providing material information in the short-wave infrared (SWIR) range where the reflection spectrum characteristics are typically very sensitive to material properties. However, previous multi-spectral LiDAR systems utilized band-pass filters or complex dispersive optical systems and even required multiple photodetectors, adding complexity and cost. In this work, we propose a time-division-multiplexing (TDM) based multi-spectral LiDAR system for semantic object inference by the simultaneous acquisition of spatial and spectral information. By utilizing the TDM method, we enable the simultaneous acquisition of spatial and spectral information as well as a TOF based distance map with minimized optical loss using only a single photodetector. Our LiDAR system utilizes nanosecond pulses of five different wavelengths in the SWIR range to acquire sufficient material information in addition to 3D spatial information. To demonstrate the recognition performance, we map the multi-spectral image from a human hand, a mannequin hand, a fabric gloved hand, a nitrile gloved hand, and a printed human hand onto an RGB-color encoded image, which clearly visualizes spectral differences as RGB color depending on the material while having a similar shape. Additionally, the classification performance of the multi-spectral image is demonstrated with a convolution neural network (CNN) model using the full multi-spectral data set. Our work presents a compact novel spectroscopic LiDAR system, which provides increased recognition performance and thus a great potential to improve safety and reliability in autonomous driving.

Trusted Data Access Control Based on Logistics Business Collaboration Semantics

Trusted Data Access Control Based on Logistics Business Collaboration Semantics

Transductive semantic decoupling double variational inference for few-shot classification

In recent years, within the rapidly evolving landscape of deep learning technology, few-shot learning, particularly in few-shot classification, has emerged as an enticing frontier. Despite the notable achievements of deep learning in handling extensive datasets, the task of image classification remains highly demanding when faced with a limited number of annotated samples. To address this challenge, we introduce the Transductive Semantic Decoupling Double Variational Inference (TSDVI), a novel framework that employs two iteratively interacting variational networks to disentangle image information and model distributions. This approach greatly improves the model's ability to discern inter-class differences, hence enabling more effective separation of features across distinct categories. Our TSDVI approach has been extensively validated through experiments, which have shown significant performance gains. These experiments were conducted on many widely-used datasets such as miniImagenet, tiered-Imagenet, CIFAR-FS, and FC100. Particularly noteworthy is the outstanding performance gain of up to 30% on the 1-shot task within the FC100 dataset. These practical results strongly emphasize the effectiveness of the TSDVI model and its promise in few-shot classification. Code is available at: https://github.com/zjh1015/tsdvi.

SEIR‐driven semantic integration framework: Internet of Things‐enhanced epidemiological surveillance in COVID‐19 outbreaks using recurrent neural networks

AbstractWith the current COVID‐19 pandemic, sophisticated epidemiological surveillance systems are more important than ever because conventional approaches have not been able to handle the scope and complexity of this global emergency. In response to this challenge, the authors present the state‐of‐the‐art SEIR‐Driven Semantic Integration Framework (SDSIF), which leverages the Internet of Things (IoT) to handle a variety of data sources. The primary innovation of SDSIF is the development of an extensive COVID‐19 ontology, which makes unmatched data interoperability and semantic inference possible. The framework facilitates not only real‐time data integration but also advanced analytics, anomaly detection, and predictive modelling through the use of Recurrent Neural Networks (RNNs). By being scalable and flexible enough to fit into different healthcare environments and geographical areas, SDSIF is revolutionising epidemiological surveillance for COVID‐19 outbreak management. Metrics such as Mean Absolute Error (MAE) and Mean sqḋ Error (MSE) are used in a rigorous evaluation. The evaluation also includes an exceptional R‐squared score, which attests to the effectiveness and ingenuity of SDSIF. Notably, a modest RMSE value of 8.70 highlights its accuracy, while a low MSE of 3.03 highlights its high predictive precision. The framework's remarkable R‐squared score of 0.99 emphasises its resilience in explaining variations in disease data even more.

Real-time semantic segmentation based on BiSeNetV2 for wild road

Abstract State-of-the-art segmentation models have shown great performance in structured road segmentation. However, these models are not suitable for the wild roads, which are highly unstructured. To tackle the problem of real-time semantic segmentation of wild roads, we propose a Multi-Information Concatenate Network based on BiSeNetV2 and construct a segmentation dataset Dalle Molle institute for artificial intelligence feature segmentation (IDSIAFS) based on Dalle Molle institute for artificial intelligence. The proposed model removes structural redundancy and optimizes the semantic branch based on BiSeNetV2. Moreover, the Dual-Path Semantic Inference Layer (TPSIL) reduces computation by designing the channel dimension of the semantic branch feature map and aggregates feature maps of different depths. Finally, the segmentation results are achieved by fusing both shallow detail information and deep semantic information. Experiments on the IDSIAFS dataset demonstrate that our proposed model achieves an 89.5% Intersection over Union. The comparative experiments on Cityscapes and India driving dataset benchmarks show that proposed model achieves good inference accuracy and faster inference speed.

FTUNet: A Feature-Enhanced Network for Medical Image Segmentation Based on the Combination of U-Shaped Network and Vision Transformer

Semantic Segmentation has been widely used in a variety of clinical images, which greatly assists medical diagnosis and other work. To address the challenge of reduced semantic inference accuracy caused by feature weakening, a pioneering network called FTUNet (Feature-enhanced Transformer UNet) was introduced, leveraging the classical Encoder-Decoder architecture. Firstly, a dual-branch Encoder is proposed based on the U-shaped structure. In addition to employing convolution for feature extraction, a Layer Transformer structure (LTrans) is established to capture long-range dependencies and global context information. Then, an Inception structural module focusing on local features is proposed at the Bottleneck, which adopts the dilated convolution to amplify the receptive field to achieve deeper semantic mining based on the comprehensive information brought by the dual Encoder. Finally, in order to amplify feature differences, a lightweight attention mechanism of feature polarization is proposed at Skip Connection, which can strengthen or suppress feature channels by reallocating weights. The experiment is conducted on 3 different medical datasets. A comprehensive and detailed comparison was conducted with 6 non-U-shaped models, 5 U-shaped models, and 3 Transformer models in 8 categories of indicators. Meanwhile, 9 kinds of layer-by-layer ablation and 4 kinds of other embedding attempts are implemented to demonstrate the optimal structure of the current FTUNet.

Semantic segmentation of satellite images with different building types using deep learning methods

In this study, using deep learning-based semantic segmentation methods, an automatic building segmentation application was carried out with a remote sensing image on a sample area covering a small part of Istanbul. In this context, firstly, fully convolutional networks, semantic segmentation inference principles, and open-source building datasets presented to the public were examined. Within the scope of the study, the IST building dataset containing examples from 5 different building type classes were created using very high resolution Pleiades satellite images. Then, building segmentation training was carried out on UNet and UNet++ architectures with this dataset. Segmentation success was compared between the models obtained after the training and the building classes according to their types. Experimental results showed that the UNET and UNet++ architecture IOU metric achieved segmentation success of 0.9167 and 0.9150 for Industrial class, 0.8124 and 0.8175 for Adjacent class, 0.8459 and 0.8446 for Housing-Villa class, 0.7629 and 0.7477 for Slum class, 0.6697 and 0.6140 for Other class. Finally, building segmentation difficulties arising from the types of buildings have been identified, and suggestions have been made to overcome this problem.

Interlinking BIM and GIS data for a semantic pedestrian network and applications in high-density cities

In high-density cities, pedestrians frequently traverse many publicly accessible indoor spaces, such as metro stations and footbridges, which seamlessly connect with outdoor sidewalks, forming indoor-outdoor combined pedestrian networks. However, the information island phenomenon hinders these connections in the digital world because outdoor sidewalks and indoor spaces are modelled using Geographical Information Systems (GIS) and Building Information Modelling (BIM) technologies, respectively, which challenges obtaining integrated information for intelligent pedestrian services. This study presents an approach for interlinking BIM and GIS data for a semantic pedestrian network using semantic web technologies. The proposed approach automatically converts BIM and GIS data into linked data and establishes interlinkages between BIM and GIS data through semantic queries and inferences. The resulting semantic pedestrian network based on integrated linked data graphs forms a knowledge base, including topological and geometrical information and abundant semantic information from BIM and GIS datasets. The application potential of the semantic pedestrian network is demonstrated through information query and semantic route planning. This research contributes to establishing indoor-outdoor combined semantic pedestrian networks ensuring seamless data integration.

Learning SAR-Optical Cross Modal Features for Land Cover Classification

Synthetic aperture radar (SAR) and optical images provide highly complementary ground information. The fusion of SAR and optical data can significantly enhance semantic segmentation inference results. However, the fusion methods for multimodal data remains a challenge for current research due to significant disparities in imaging mechanisms from diverse sources. Our goal was to bridge the significant gaps between optical and SAR images by developing a dual-input model that utilizes image-level fusion. To improve most existing state-of-the-art image fusion methods, which often assign equal weights to multiple modalities, we employed the principal component analysis (PCA) transform approach. Subsequently, we performed feature-level fusion on shallow feature maps, which retain rich geometric information. We also incorporated a channel attention module to highlight channels rich in features and suppress irrelevant information. This step is crucial due to the substantial similarity between SAR and optical images in shallow layers such as geometric features. In summary, we propose a generic multimodal fusion strategy that can be attached to most encoding–decoding structures for feature classification tasks, designed with two inputs. One input is the optical image, and the other is the three-band fusion data obtained by combining the PCA component of the optical image with the SAR. Our feature-level fusion method effectively integrates multimodal data. The efficiency of our approach was validated using various public datasets, and the results showed significant improvements when applied to several land cover classification models.

Toward Automated Field Semantics Inference for Binary Protocol Reverse Engineering

Toward Automated Field Semantics Inference for Binary Protocol Reverse Engineering

Integrating Large Language Model, EEG, and Eye-Tracking for Word-Level Neural State Classification in Reading Comprehension.

With the recent proliferation of large language models (LLMs), such as Generative Pre-trained Transformers (GPT), there has been a significant shift in exploring human and machine comprehension of semantic language meaning. This shift calls for interdisciplinary research that bridges cognitive science and natural language processing (NLP). This pilot study aims to provide insights into individuals' neural states during a semantic inference reading-comprehension task. We propose jointly analyzing LLMs, eye-gaze, and electroencephalographic (EEG) data to study how the brain processes words with varying degrees of relevance to a keyword during reading. We also use feature engineering to improve the fixation-related EEG data classification while participants read words with high versus low relevance to the keyword. The best validation accuracy in this word-level classification is over 60% across 12 subjects. Words highly relevant to the inference keyword received significantly more eye fixations per word: 1.0584 compared to 0.6576, including words with no fixations. This study represents the first attempt to classify brain states at a word level using LLM-generated labels. It provides valuable insights into human cognitive abilities and Artificial General Intelligence (AGI), and offers guidance for developing potential reading-assisted technologies.

Unveiling Typographic Artistry in Aravind Adiga's The White Tiger: A Critical Examination of Deeper Semantic Inference

Unveiling Typographic Artistry in Aravind Adiga's The White Tiger: A Critical Examination of Deeper Semantic Inference

Classification of Urban Surface Elements by Combining Multisource Data and Ontology

The rapid pace of urbanization and increasing demands for urban functionalities have led to diversification and complexity in the types of urban surface elements. The conventional approach of relying solely on remote sensing imagery for urban surface element extraction faces emerging challenges. Data-driven techniques, including deep learning and machine learning, necessitate a substantial number of annotated samples as prerequisites. In response, our study proposes a knowledge-driven approach that integrates multisource data with ontology to achieve precise urban surface element extraction. Within this framework, components from the EIONET Action Group on Land Monitoring in Europe matrix serve as ontology primitives, forming a shared vocabulary. The semantics of surface elements are deconstructed using these primitives, enabling the creation of specific descriptions for various types of urban surface elements by combining these primitives. Our approach integrates multitemporal high-resolution remote sensing data, network big data, and other heterogeneous data sources. It segments high-resolution images into individual patches, and for each unit, urban surface element classification is accomplished through semantic rule-based inference. We conducted experiments in two regions with varying levels of urban scene complexity, achieving overall accuracies of 93.03% and 97.35%, respectively. Through this knowledge-driven approach, our proposed method significantly enhances the classification performance of urban surface elements in complex scenes, even in the absence of sample data, thereby presenting a novel approach to urban surface element extraction.