Visual Semantic Analysis Research Articles

An Interactive Image Segmentation Method Based on Multi-Level Semantic Fusion.

Understanding and analyzing 2D/3D sensor data is crucial for a wide range of machine learning-based applications, including object detection, scene segmentation, and salient object detection. In this context, interactive object segmentation is a vital task in image editing and medical diagnosis, involving the accurate separation of the target object from its background based on user annotation information. However, existing interactive object segmentation methods struggle to effectively leverage such information to guide object-segmentation models. To address these challenges, this paper proposes an interactive image-segmentation technique for static images based on multi-level semantic fusion. Our method utilizes user-guidance information both inside and outside the target object to segment it from the static image, making it applicable to both 2D and 3D sensor data. The proposed method introduces a cross-stage feature aggregation module, enabling the effective propagation of multi-scale features from previous stages to the current stage. This mechanism prevents the loss of semantic information caused by multiple upsampling and downsampling of the network, allowing the current stage to make better use of semantic information from the previous stage. Additionally, we incorporate a feature channel attention mechanism to address the issue of rough network segmentation edges. This mechanism captures richer feature details from the feature channel level, leading to finer segmentation edges. In the experimental evaluation conducted on the PASCAL Visual Object Classes (VOC) 2012 dataset, our proposed interactive image segmentation method based on multi-level semantic fusion demonstrates an intersection over union (IOU) accuracy approximately 2.1% higher than the currently popular interactive image segmentation method in static images. The comparative analysis highlights the improved performance and effectiveness of our method. Furthermore, our method exhibits potential applications in various fields, including medical imaging and robotics. Its compatibility with other machine learning methods for visual semantic analysis allows for integration into existing workflows. These aspects emphasize the significance of our contributions in advancing interactive image-segmentation techniques and their practical utility in real-world applications.

Research on Semantic Extraction of Online Mathematics Guidance Videos in Colleges and Universities Based on Intelligent Information Technology

This article proposes a general solution for multi-granularity semantic analysis and extraction of video data based on the online instructional video of mathematics in colleges and universities of intelligent information technology. In this scheme, multi-level semantic analysis and multi-modal information fusion technology are unified and applied in the same model. This paper first proposes a method for detecting the gradual change of shots based on statistical distribution, and uses a key frame selection strategy with temporal semantic context constraints to represent the temporal content. After basic visual semantic recognition, a hierarchical approach is obtained. The multi-granularity visual semantic analysis extraction framework then uses the sound spectrum obtained by the time-frequency transformation as the observable feature, and constructs a hidden Markov model for semantic recognition of mathematical videos, which improves the efficiency by 7.93%.

Neighboring Algorithm for Visual Semantic Analysis toward GAN-Generated Pictures.

Generative adversarial network (GAN)-guided visual quality evaluation means scoring GAN-propagated portraits to quantify the degree of visual distortions. In general, there are very few image- and character-evaluation algorithms generated by GAN, and the algorithm's athletic ability is not capable. In this article, we proposed a novel image ranking algorithm based on the nearest neighbor algorithm. It can obtain automatic and extrinsic evaluation of GAN procreate images using an efficient evaluation technique. First, with the support of the artificial neural network, the boundaries of the variety images are extracted to form a homogeneous portrait candidate pool, based on which the comparison of product copies is restricted. Subsequently, with the support of the K-nearest neighbors algorithm, from the unified similarity candidate pool, we extract the most similar concept of K-Emperor to the generated portrait and calculate the portrait quality score accordingly. Finally, the property of generative similarity that produced by the GAN models are trained on a variety of classical datasets. Comprehensive experimental results have shown that our algorithm substantially improves the efficiency and accuracy of the natural evaluation of pictures generated by GAN. The calculated metric is only 1/9–1/28 compared to the other methods. Meanwhile, the objective evaluation of the GAN and human consistency has increased by more than 80% in line with human visual perception.

The Use of Convolutional Neural Networks in the Prediction of Invasive Ductal Carcinoma in Histological Images of Breast Cancer.

This paper presents a deep learning approach for automatic detection and visual analysis of Invasive Ductal Carcinoma (IDC) tissue regions. The method proposed in this work is a convolutional neural network (CNN) for visual semantic analysis of tumor regions for diagnostic support. Detection of IDC is a time-consuming and challenging task, mainly because a pathologist needs to examine large tissue regions to identify areas of malignancy. Deep Learning approaches are particularly suitable for dealing with this type of problem, especially when many samples are available for training, ensuring high quality of the learned features by the classifier and, consequently, its generalization capacity. A 3-hidden-layer CNN with data balancing reached both accuracy and F1-Score of 0.85 and outperforming other approaches from the literature. Thus, the proposed method in this article can serve as a support tool for the identification of invasive breast cancer.

Systematic Review of Computer Vision Semantic Analysis in Socially Assistive Robotics

The simultaneous surges in the research on socially assistive robotics and that on computer vision can be seen as a result of the shifting and increasing necessities of our global population, especially towards social care with the expanding population in need of socially assistive robotics. The merging of these fields creates demand for more complex and autonomous solutions, often struggling with the lack of contextual understanding of tasks that semantic analysis can provide and hardware limitations. Solving those issues can provide more comfortable and safer environments for the individuals in most need. This work aimed to understand the current scope of science in the merging fields of computer vision and semantic analysis in lightweight models for robotic assistance. Therefore, we present a systematic review of visual semantics works concerned with assistive robotics. Furthermore, we discuss the trends and possible research gaps in those fields. We detail our research protocol, present the state of the art and future trends, and answer five pertinent research questions. Out of 459 articles, 22 works matching the defined scope were selected, rated in 8 quality criteria relevant to our search, and discussed in depth. Our results point to an emerging field of research with challenging gaps to be explored by the academic community. Data on database study collection, year of publishing, and the discussion of methods and datasets are displayed. We observe that the current methods regarding visual semantic analysis show two main trends. At first, there is an abstraction of contextual data to enable an automated understanding of tasks. We also observed a clearer formalization of model compaction metrics.

A SELF-BALANCED CLUSTERING TREE FOR SEMANTIC-BASED IMAGE RETRIEVAL

The image retrieval and semantic extraction play an important role in the multimedia systems such as geographic information system, hospital information system, digital library system, etc. Therefore, the research and development of semantic-based image retrieval (SBIR) systems have become extremely important and urgent. Major recent publications are included covering different aspects of the research in this area, including building data models, low-level image feature extraction, and deriving high-level semantic features. However, there is still no general approach for semantic-based image retrieval (SBIR), due to the diversity and complexity of high-level semantics. In order to improve the retrieval accuracy of SBIR systems, our focus research is to build a data structure for finding similar images, from that retrieving its semantic. In this paper, we proposed a data structure which is a self-balanced clustering tree named C-Tree. Firstly, a method of visual semantic analysis relied on visual features and image content is proposed on C-Tree. The building of this structure is created based on a combination of methods including hierarchical clustering and partitional clustering. Secondly, we design ontology for the image dataset and create the SPARQL (SPARQL Protocol and RDF Query Language) query by extracting semantics of image. Finally, the semantic-based image retrieval on C-Tree (SBIR\_CT) model is created hinging on our proposal. The experimental evaluation 20,000 images of ImageCLEF dataset indicates the effectiveness of the proposed method. These results are compared with some of recently published methods on the same dataset and demonstrate that the proposed method improves the retrieval accuracy and efficiency.

A SELF-BALANCED CLUSTERING TREE FOR SEMANTIC-BASED IMAGE RETRIEVAL

The image retrieval and semantic extraction play an important role in the multimedia systems such as geographic information system, hospital information system, digital library system, etc. Therefore, the research and development of semantic-based image retrieval (SBIR) systems have become extremely important and urgent. Major recent publications are included covering different aspects of the research in this area, including building data models, low-level image feature extraction, and deriving high-level semantic features. However, there is still no general approach for semantic-based image retrieval (SBIR), due to the diversity and complexity of high-level semantics. In order to improve the retrieval accuracy of SBIR systems, our focus research is to build a data structure for finding similar images, from that retrieving its semantic. In this paper, we proposed a data structure which is a self-balanced clustering tree named C-Tree. Firstly, a method of visual semantic analysis relied on visual features and image content is proposed on C-Tree. The building of this structure is created based on a combination of methods including hierarchical clustering and partitional clustering. Secondly, we design ontology for the image dataset and create the SPARQL (SPARQL Protocol and RDF Query Language) query by extracting semantics of image. Finally, the semantic-based image retrieval on C-Tree (SBIR\_CT) model is created hinging on our proposal. The experimental evaluation 20,000 images of ImageCLEF dataset indicates the effectiveness of the proposed method. These results are compared with some of recently published methods on the same dataset and demonstrate that the proposed method improves the retrieval accuracy and efficiency.

Special issue on “visual semantic analysis with weak supervision”

Special issue on “visual semantic analysis with weak supervision”

Contribution of pictorial help to the understanding of medical prescriptions in elderly adults and in patients with Alzheimer's disease

The effects on comprehension of presenting medical prescription information in a pictorial format were compared to a verbal format in healthy old adults and in patients suffering from Alzheimer disease (AD). Participants received medical prescription instructions in one of two formats: with verbal instructions only, or with verbal instructions associated with pictorial representations. The results indicated that comprehension for prescription information of healthy old subjects and AD patients was facilitated by the combined presentation. Furthermore, the pictorial instructions appeared acting on the capacity to work out a specific medication schemata, that is to say on their real, operational, understanding. These results suggest that analogical pictorial format leads more directly to the medical prescriptions' situation model and alleviates the particularization deficiency of the therapeutic schemata, independently of global cognitive functioning, visual perception, space perception and visual semantic analysis in AD. The presentation of illustrations representing medical prescriptions would allow to decrease doubts and errors, which are frequent in normal aging and even more in Alzheimer's disease, and contribute to a better medical compliency.