Spatial Relationships Of Objects Research Articles

Spatial and temporal consistency learning for monocular 6D pose estimation

Monocular 6D pose estimation is a challenging task in the field of computer vision and robotics. Many previous works only input the cropped image of single object during training and inference, aiming to remove the noise from non-object regions. However, most of these methods ignore the viewpoint and spatial relationships of objects in the scene, which are crucial for accurate pose estimation of camera. To address this issue, this paper proposes a novel multi-view and multi-object based learning strategy for monocular 6D pose estimation, which involves the consistency of object coordinate for the same object at different viewpoints and the consistency of world coordinate for different objects in the same space. In the proposed method, the spatial and temporal groups are generated to trained the monocular 6D pose estimation network. Due to the camera motion, scene images taken at different times can be regarded as images captured from different viewpoints. Therefore, a temporal consistency loss is designed to constraint the relationship of the same object at different viewpoints, while a spatial consistency loss is designed to constraint the relationship of different objects at the same space. Finally, the proposed method is verified on the public datasets. Experimental results show that the proposed method is accurate, robust, and outperforms similar state-of-the-art approaches.

Deep Generative Modeling Based on VAE-GAN for 3D Indoor Scene Synthesis

With the advancement of virtual reality and 3D game technology, the demand for high-quality 3D indoor scene generation has surged. Addressing this need, this paper presents a method leveraging a VAE-GAN-based framework to conquer two primary challenges in 3D scene representation and deep generative networks. First, we introduce a matrix representation to encode fine-grained object attributes, alongside a complete graph to implicitly capture object spatial relations—effectively encapsulating both local and global scene structures. Second, we devise a unique generative framework based on VAE-GAN and the Bayesian optimization. This framework learns a Gaussian distribution of encoded object attributes through a VAE-GAN network, allowing for sampling and decoding of the distribution to generate new object attributes. Subsequently, a U-Net is employed to learn spatial relations between objects. Lastly, the Bayesian optimization module amalgamates the generated object attributes, spatial relations, and priors learned from data, conducting global optimization to generate a logical scene layout. Experimental results on a large-scale 3D indoor scene dataset substantiate that our method effectively learns inter-object relations and generates diverse and plausible indoor scenes. Comparative experiments and user studies further validate that our method surpasses the current state-of-the-art techniques in indoor scene generation and is comparable to real training scenes.

Classification of multi-modal remote sensing images based on knowledge graph

ABSTRACT With the development of remote sensing (RS) technology, single-modal data alone has gradually become difficult to meet the requirement for high accuracy of RS image classification. As a result, multi-modal RS image classification has become a hot research topic. However, multi-modal RS image classification faces challenges in effectively utilizing advanced semantic information regarding the relationships between land cover classes and extracting discriminative features from the data. Based on this, a multi-modal RS image classification method based on knowledge graph (KG) is proposed. Firstly, graph topology is used to align the hyperspectral image (HSI) and multispectral image (MSI) features extracted by graph convolution networks. Constraint alignment is performed on different feature graphs to reduce the difficulty of fusion and the false recognition rate. Then, we use self-attention and cross-attention to purposefully fuse HSI and MSI to obtain discriminative features rich in two modal information and achieve feature weighted fusion. Finally, a KG based on object spatial relationships is constructed to obtain spatial relationships between different classes to assist in multi-modal RS image classification. The experimental results on the Houston and Ausburg datasets demonstrate that the proposed method achieves overall accuracy of 90.40% and 90.85%, respectively, both of which are more than 3% higher than existing classification methods. The results indicate that our method has better classification performance and can provide a useful reference for RS image classification research.

Robot task programming in complex task scenarios based on spatio-temporal constraint calculus

PurposeRobots face fundamental challenges in achieving reliable and stable operations for complex home service scenarios. This is one of the crucial topics of robotics methods to imitate human beings’ advanced cognitive characteristics and apply them to solve complex tasks. The purpose of this study is to enable robots to have the ability to understand the scene and task process in complex scenes and to provide a reference method for robot task programming in complex scenes.Design/methodology/approachThis paper constructs a task modeling method for robots in complex environments based on the characteristics of the perception-motor memory model of human cognition. In the aspect of episodic memory construction, the task execution process is included in the category of qualitative spatio-temporal calculus. The topology interaction of objects in a task scenario is used to define scene attributes. The task process can be regarded as changing scene attributes on a time scale. The qualitative spatio-temporal activity graphs are used to analyze the change process of the object state with time during the robot task execution. The tasks are divided according to the different values of scene attributes at different times during task execution. Based on this, in procedural memory, an object-centered motion model is developed by analyzing the changes in the relationship between objects in the scene episode by analyzing the scene changes before and after the robot performs the actions. Finally, the task execution process of the robot is constructed by alternately reconstructing episodic memory and procedural memory.FindingsTo verify the applicability of the proposed model, a scenario where the robot combines the object (one of the most common tasks in-home service) is set up. The proposed method can obtain the landscape of robot tasks in a complex environment.Originality/valueThe robot can achieve high-level task programming through the alternating interpretation of scenarios and actions. The proposed model differs from traditional methods based on geometric or physical feature information. However, it focuses on the spatial relationship of objects, which is more similar to the cognitive mechanism of human understanding of the environment.

Fenomena Literasi Spasial pada Siswa Kelas VII SMP Materi Bangun Datar

Spatial literacy is an important component that students must master. Spatial literacy supports students' understanding of flat shapes which are closely related to the real world. This study aims to describe the phenomenon of spatial literacy in student studies on flat shapes. This research design uses a descriptive with a qualitative approach. Data collection techniques used in this study were observation, pretest & posttest, and interviews. The research location in this study was VII Minqhotrotul Ulum Gumukmas Middle School. The subjects in this study consisted of 3 students who had high mathematical abilities and 2 students who had low mathematical abilities. The results of this study indicate that students who have high abilities can sketch spatial objects through visual images based on contextual problems, identify questions and statements of mathematical models and relate concepts and spatial object relationships in calculation techniques and are able to express ideas related to spatial object relationships through oral and written . Whereas students who have low mathematical abilities have known that a phenomenon cannot create spatial objects through visual images based on contextual problems, is unable to express ideas related to spatial object relationships through speech and writing, but can identify questions and statements of mathematical models, but cannot relate concepts and spatial object relationships in calculation techniques. so from this phenomenon it can be concluded that students who have high mathematical abilities have fulfilled the three aspects of spatial literacy indicators, namely the aspects of Visualization, Reasoning, and Communication, and students who have low mathematical abilities do not fulfill the three aspects of spatial literacy indicators. Keywords: Phenomena, Spatial Literacy, Flat Shape Study.

Interactive and incremental learning of spatial object relations from human demonstrations.

Humans use semantic concepts such as spatial relations between objects to describe scenes and communicate tasks such as "Put the tea to the right of the cup" or "Move the plate between the fork and the spoon." Just as children, assistive robots must be able to learn the sub-symbolic meaning of such concepts from human demonstrations and instructions. We address the problem of incrementally learning geometric models of spatial relations from few demonstrations collected online during interaction with a human. Such models enable a robot to manipulate objects in order to fulfill desired spatial relations specified by verbal instructions. At the start, we assume the robot has no geometric model of spatial relations. Given a task as above, the robot requests the user to demonstrate the task once in order to create a model from a single demonstration, leveraging cylindrical probability distribution as generative representation of spatial relations. We show how this model can be updated incrementally with each new demonstration without access to past examples in a sample-efficient way using incremental maximum likelihood estimation, and demonstrate the approach on a real humanoid robot.

Studying spatial orientation and constructive praxis disorders in normally developing and mentally retarded children aged 8-11 years

The ability to orientate oneself in space is one of the basic requirements for the harmonious development of a person. The problem of human orientation in space is very complex. It includes both ideas about the size and shape of objects and the ability to distinguish the location of objects in space, and understanding different spatial relationships. The concept of spatial orientation includes the evaluation of distances, sizes, shapes, mutual position of objects and their position relative to the orienting object. Spatial orientation is significantly impaired in mentally handicapped children, since the impairment of mental development not only limits their motor activity, but also negatively affects the development of time representations, musculoskeletal functions and leads to a decrease in compensatory processes. The purpose of this study is to identify spatial orientation and constructive praxis disorders in normally developing and mentally retarded children aged 8-11 years. Materials and Methods. The methodological basis of this research investigation was scholarly literature devoted to studying nervous functions in children with mental development disorders and interhemispheric asymmetry of the brain, regulation of these functions, identifying difficulties, modeling the system of work with these children and constructive praxis disorders in normally developing and mentally retarded children aged 8-11 years (Boguslavskaya, Miroshnichenko, 2019, Maryutina, Yermolaev 2001, Reuter-Lorenz and others, 2000, Azatyan2022). The study was conducted in the research laboratory of the Faculty of Special and Inclusive Education at the Armenian State Pedagogical University after Kh. Abovyan and research laboratory in Armenian state institute of physical culture and sport. A total of 131 children aged between 8 and 11 years participated in the study, including 73 healthy schoolchildren and 58 children with mild mental retardation. The degree of intellectual disability was assessed based on medical indications, degree of social adjustment, level of intellectual functioning, and learning outcomes in special education classes. Each category of participants was divided into 2 age groups: 8-9 years old and 10-11 years old. Results. The article discusses the main issues of studies of spatial orientation and constructive praxis in normally developing and mentally retarded children aged between 8 and 11 years, provides a scientific rationale for the problem, taking into account previous research investigations. However, the problem of mentally retarded and normally developing children in this age group remains virtually unexplored. Analysis of research findings allows the authors to identify and summarize the following important points: - mentally handicapped primary school children lag behind their healthy peers in all indicators examined; - 8- and 9-year-olds have less developed practical spatial orientation skills than their older peers; - Intellectually disabled peers have underdeveloped spatial orientation skills and constructive practice, making it difficult to understand spatial relationships between objects. The experimental study of task performance revealed interesting data, which will serve as a basis for developing the necessary means, methods and conditions for the development of elementary practical orientation in mentally disabled primary schoolchildren. The authors have uncovered violations of spatial orientation and constructive praxis in normally developing and intellectually disabled children aged 8-11 years that highlight the limited and incomplete verbalization of spatial relationships of objects in intellectually disabled children. The listed errors in general are reduced to the use by mentally retarded schoolchildren of imprecise, vague spatial characteristics instead of more differentiated ones. Conclusions. Thus, the results of the conducted experimental research testify to the insufficient development of the elementary practical orientation and understanding of the spatial relationships of objects in mentally retarded children of primary school age.

Depth-Aware and Semantic Guided Relational Attention Network for Visual Question Answering

Visual relationship understanding plays an indispensable role in grounded language tasks like visual question answering (VQA), which often requires precisely reasoning about relations among objects depicted in the given question. However, prior works generally suffer from the deficiencies as follows, (1) <b>spatial-relation inference ambiguity</b>, it is challenging to accurately estimate the distance of a pair of visual objects in 2D space if there is a visual-overlap between their 2D bounding-boxes, and (2) language-visual relational alignment missing, it is insufficient to generate a high-quality answer to the question if there is a lack of alignment in the language-visual relations of objects during fusion, even using a powerful fusion model like Transformer. To this end, we first model the spatial relation of a pair of objects in 3D space by augmenting the original 2D bounding-box with 1D depth information, and then propose a novel model named <b><u>D</u></b>epth-aware <b><u>S</u></b>emantic <b><u>G</u></b>uided Relational <b><u>A</u></b>ttention Network (DSGANet), to explicitly exploit the formed 3D spatial relations of objects in an intra-/inter-modality manner for precise relational alignment. Extensive experiments conducted on the benchmarks (VQA v2.0 and GQA) demonstrate DSGANet achieves competitive performance compared to pretrained and non-pretrained models, such as 72.7&#x0025; vs. 74.6&#x0025; based on the learned grid features on VQA v2.0.

Metaphorical Approach in Interior Architecture Education

In this study, the project studio, which is the foundation of interior design education, has been designed using the metaphor method, one of the intellectual methods applied in education. The metaphor method is a technique used to make difficult-to-understand abstract concepts concrete and helps design students develop their critical thinking and creative skills. In this case, design, which is the verbalized form of abstract thought, is limited to an object and the responsibilities brought by this object. From this point of view, an experimental study of metaphorical methods for establishing spatial relations of objects was carried out in a 14-week design studio with the 2nd year students of the Department of Interior Architecture, Faculty of Architecture and Design, Selcuk University. The purpose of the studio work is to draw the boundaries of interior architecture students in their search for solutions to design problems, and to enable them to come up with creative designs while making functional and formal decisions. In this context, a metaphorical approach was used in the context of object-space relations to guide students’ formal decisions in the functional solutions of the problems that arise in the interior design process. After working in the studio, the students have developed a working doctrine that will guide their interior design studio projects using the designed method, allow them to create unique forms and shapes, enable them to translate abstract ideas into concrete space and make it possible for them to design at various scales, from equipment to space design. The method is at a level that will set an example for students, designers and academicians working in the design discipline.

Research of Models of Topological Relations of Spatial Objects

In map production it is necessary to keep the spatial relationships between map objects. Generalization is the simplification performed on geographical data when decreasing its representation scale. It is a common practice to simplify each type of spatial objects independently (administrative boundaries first, then road network, hydrographic network, etc.). During the process some spatial conflicts, which require manual correction, arise inevitably. The generalization automation still remains an open issue for data producers and users. Many researchers are working to achieve a higher level of automation. In order to detect the spatial conflicts a refined description of spatial relationships is needed.The paper analyzes models of describing topological relationships of spatial objects: the nine intersections model, the topological chain model and the E-WID model. Each considered model allows to take into account some relations between objects, but does not allow to transfer them exactly. As a result, the task of developing a model of relations preserving topology is relevant. We have proposed an improved model of nine intersections, which takes into account the topological conflict that occurs when a point object is located next to a simplified line. Line simplification is one of the most requested actions in map production and generalization. When the mesh covered the map inside the cell there can be points, line segments and polygonal topological objects, which, if the cell is rather small, are polyline objects. Thus, the issue of simplification of topological objects within a cell is reduced to the issue of simplification of linear objects (polylines). The developed algorithm is planned to be used to solve the problem of consistent generalization of spatial data. The ideas outlined in this article will form the basis of a new index of spatial data that preserves their topological relationships.

Study on Spatial Geometric Similarity Based on Conformal Geometric Algebra.

The study of spatial geometric similarity plays a significant role in spatial data retrieval. Many researchers have examined spatial geometric similarity, which is useful for spatial analysis and data retrieval. However, the majority of them focused on objects of the same type. Methods to support the spatial geometric similarity computation for different types of objects are rare, a systematic theory index has not been developed yet, and there has not been a comprehensive computational model of spatial geometric similarity. In this study, we conducted an analysis of the spatial geometric similarity computation based on conformal geometric algebra (CGA), which has certain advantages in the quantitative computation of the measurement information of spatial objects and the qualitative judgment of the topological relations of spatial objects. First, we developed a unified expression model for spatial geometric scenes, integrating shapes of objects and spatial relations between them. Then, we established a model for the spatial geometric similarity computation under various geographical circumstances to provide a novel approach for spatial geometric similarity research. Finally, the computation model was verified through a case study. The study of spatial geometric similarity sheds light on spatial data retrieval, which has scientific significance and practical value.

A Spatial Relationship Preserving Adversarial Network for 3D Reconstruction from a Single Depth View

Recovering the geometry of an object from a single depth image is an interesting yet challenging problem. While previous learning based approaches have demonstrated promising performance, they don’t fully explore spatial relationships of objects, which leads to unfaithful and incomplete 3D reconstruction. To address these issues, we propose a Spatial Relationship Preserving Adversarial Network (SRPAN) consisting of 3D Capsule Attention Generative Adversarial Network (3DCAGAN) and 2D Generative Adversarial Network (2DGAN) for coarse-to-fine 3D reconstruction from a single depth view of an object. Firstly, 3DCAGAN predicts the coarse geometry using an encoder-decoder based generator and a discriminator. The generator encodes the input as latent capsules represented as stacked activity vectors with local-to-global relationships (i.e., the contribution of components to the whole shape), and then decodes the capsules by modeling local-to-local relationships (i.e., the relationships among components) in an attention mechanism. Afterwards, 2DGAN refines the local geometry slice-by-slice, by using a generator learning a global structure prior as guidance, and stacked discriminators enforcing local geometric constraints. Experimental results show that SRPAN not only outperforms several state-of-the-art methods by a large margin on both synthetic datasets and real-world datasets, but also reconstructs unseen object categories with a higher accuracy.

Технология извлечения геоатрибутированных сущностей для визуального представления пространственной связности объектов на основе автоматизированной генерации картосхем

This paper considers the problem of extracting geoattributed entities from natural language texts to visualize the spatial relations of geographical objects. For visualization we use the technology of automated generation of schematic maps as subject-oriented components of geographic information systems. The paper describes the information technology that allows extracting geoattributed entities from natural language texts by combining several approaches. These are the neural network approach, the rule-based approach and the approach based on the use of lexico-syntactic patterns for the analysis of natural language texts. For data visualization we propose to use automated geocoding tools in conjunction with the capabilities of modern geographic information systems. The result of this technology is a cartogram that displays the spatial relations of the objects mentioned in the text.

Spatial Orientation Of Objects Based On Processing Of A Natural Language Text For Storyboard Generation

The article is devoted to our approaches to processing text in natural language to clarify the specific spatial relationship of objects and three-dimensional frame-by-frame visualization. The proposed approach allows us to show how the explicit constraints of the extracted spatial relationships affect and makes it possible to create possible layouts of objects on the scene. Natural language interpretations for spatial knowledge can generate three-dimensional scenes, which in turn are necessary to translate the scriptwriter's intent into the design of video games. The work also takes into account the rules of directing to create successful shots. Among them, accounting for the plan, camera rotation, as well as compositional nuances.

Synthesis of Prostate MR Images for Classification Using Capsule Network-Based GAN Model.

Prostate cancer remains a major health concern among elderly men. Deep learning is a state-of-the-art technique for MR image-based prostate cancer diagnosis, but one of major bottlenecks is the severe lack of annotated MR images. The traditional and Generative Adversarial Network (GAN)-based data augmentation methods cannot ensure the quality and the diversity of generated training samples. In this paper, we have proposed a novel GAN model for synthesis of MR images by utilizing its powerful ability in modeling the complex data distributions. The proposed model is designed based on the architecture of deep convolutional GAN. To learn the more equivariant representation of images that is robust to the changes in the pose and spatial relationship of objects in the images, the capsule network is applied to replace CNN used in the discriminator of regular GAN. Meanwhile, the least squares loss has been adopted for both the generator and discriminator in the proposed GAN to address the vanishing gradient problem of sigmoid cross entropy loss function in regular GAN. Extensive experiments are conducted on the simulated and real MR images. The results demonstrate that the proposed capsule network-based GAN model can generate more realistic and higher quality MR images than the compared GANs. The quantitative comparisons show that among all evaluated models, the proposed GAN generally achieves the smallest Kullback–Leibler divergence values for image generation task and provides the best classification performance when it is introduced into the deep learning method for image classification task.

A Location-and-Form-Based Distance for Geographical Analysis

Location and geometric form constitute a fundamental basis for the characterization of objects in space. The traditional distance-based geographical analysis of objects, however, usually ignores information associated with their forms. In this article, we propose a location-and-form-based distance to simultaneously take into account these basic characteristics. For substantiation, the significance of the proposed distance is examined with respect to its methodological contributions and applicability. In terms of methodology, we use pattern analysis by the L statistic as an example to show the form effect on the conventional geographical analysis, which is based solely on the location-based distance, and show how to generalize point distance–based analyses to the analysis of objects with forms. With respect to applicability, we demonstrate the capability of our proposed distance in improving the performance of matching buildings in OpenStreetMap and the corresponding standard reference of an area. It is also applicable to other real-life problems in which object forms are involved. Generally, the location-and-form-based distance and the associated methods can give us a new perspective on the conceptualization of distance. The proposed distance can be further extended to include other object attributes to study spatial relationships of objects based on a general notion of distance. Therefore, the proposed distance is a powerful concept that can comprehensively reveal the multifaceted nature of geographical relationships. This study advances the frontier of theoretical and applied research in geography where distance plays an important role.

Joint Input and Output Space Learning for Multi-Label Image Classification

Multi-label image classification aims to predict the labels associated with a given image. While most existing methods utilize unified image representations, extracting label-specific features through input space learning would improve the discriminative power of the learned features. On the other hand, most feature learning studies often ignore the learning in the output label space, although taking advantage of label correlations can boost the classification performance. In this paper, we propose a deep learning framework that incorporates flexible modules which can learn from both input and output spaces for multi-label image classification. For the input space learning, we devise a label-specific feature pooling method to refine convolutional features for obtaining features specific to each label. For the output space learning, we design a Two-Stream Graph Convolutional Network (TSGCN) to learn multi-label classifiers by mapping spatial object relationships and semantic label correlations. More specifically, we build object spatial graphs to characterize the spatial relationships among objects in an image, which supplements the label semantic graphs modelling the semantic label correlations. Experimental results on two popular benchmark datasets (i.e., Pascal VOC and MS-COCO) show that our proposed method achieves superior performance over the state-of-the-arts.

Spatial Relational Attention Using Fully Convolutional Networks for Image Caption Generation

Attention-based encoder–decoder framework has greatly improved image caption generation tasks. The attention mechanism plays a transitional role by transforming static image features into sequential captions. To generate reasonable captions, it is of great significance to detect spatial characteristics of images. In this paper, we propose a spatial relational attention approach to consider spatial positions and attributes. Image features are firstly weighted by the attention mechanism. Then they are concatenated with contextual features to form a spatial–visual tensor. The tensor is feature extracted by a fully convolutional network to produce visual concepts for the decoder network. The fully convolutional layers maintain spatial topology of images. Experiments conducted on the three benchmark datasets, namely Flickr8k, Flickr30k and MSCOCO, demonstrate the effectiveness of our proposed approach. Captions generated by the spatial relational attention method precisely capture spatial relations of objects.

An Approach to Global Illumination Calculation Based on Hybrid Cone Tracing

For 3D geographic information systems (GIS) or video game systems, global illumination (GI) effect can greatly improve the understanding of the volumetric structure and the spatial relationships of objects in the scene. However, GI effect is computationally expensive. It requires not only complex visibility computations between arbitrary points but also the integration computation over a large number of directions. These two computations become extremely difficult tasks for the real-time or interactive algorithms of the dynamic scene. This paper proposes a lightweight GI calculation approach built upon a hybrid cone tracing algorithm with which to approximate the GI effect of the open world scene in real time. First, the 3D scene is divided into two types: complex meshes and height-field meshes. Second, the corresponding lightmaps of the two meshes are generated and mipmaped, respectively. GPU hardware acceleration technology is used to do the calculation efficiently. Finally, a hybrid cone tracing method is performed on the GPU to gather the indirect lighting information for each shaded point. The method first carries out the cone-scene intersection test and then carries out the light sampling and accumulation calculation. All tracing calculations are based on the hybrid lightmap representation. In addition, the experiment results show the effectiveness of our method. Compared with the 3D texture VCT method, memory consumption can be reduced by up to 80% in our experiment.

Learning Object-Action Relations from Bimanual Human Demonstration Using Graph Networks

Recognizing human actions is a vital task for a humanoid robot, especially in domains like programming by demonstration. Previous approaches on action recognition primarily focused on the overall prevalent action being executed, but we argue that bimanual human motion cannot always be described sufficiently with a single action label. We present a system for framewise action classification and segmentation in bimanual human demonstrations. The system extracts symbolic spatial object relations from raw RGB-D video data captured from the robot's point of view in order to build graph-based scene representations. To learn object-action relations, a graph network classifier is trained using these representations together with ground truth action labels to predict the action executed by each hand. We evaluated the proposed classifier on a new RGB-D video dataset showing daily action sequences focusing on bimanual manipulation actions. It consists of 6 subjects performing 9 tasks with 10 repetitions each, which leads to 540 video recordings with 2 hours and 18 minutes total playtime and per-hand ground truth action labels for each frame. We show that the classifier is able to reliably identify (action classification macro F1-score of 0.86) the true executed action of each hand within its top 3 predictions on a frame-by-frame basis without prior temporal action segmentation.