Composite Scene Research Articles

Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation.

Visual Question Answering on 3D Point Cloud (VQA-3D) is an emerging yet challenging field that aims at answering various types of textual questions given an entire point cloud scene. To tackle this problem, we propose the CLEVR3D, a large-scale VQA-3D dataset consisting of 171K questions from 8,771 3D scenes. Specifically, we develop a question engine leveraging 3D scene graph structures to generate diverse reasoning questions, covering the questions of objects' attributes (i.e., size, color, and material) and their spatial relationships. Through such a manner, we initially generated 44K questions from 1,333 real-world scenes. Moreover, a more challenging setup is proposed to remove the confounding bias and adjust the context from a common-sense layout. Such a setup requires the network to achieve comprehensive visual understanding when the 3D scene is different from the general co-occurrence context (e.g., chairs always exist with tables). To this end, we further introduce the compositional scene manipulation strategy and generate 127K questions from 7,438 augmented 3D scenes, which can improve VQA-3D models for real-world comprehension. Built upon the proposed dataset, we baseline several VQA-3D models, where experimental results verify that the CLEVR3D can significantly boost other 3D scene understanding tasks.

Targeted Memory Reactivation during Nonrapid Eye Movement Sleep Enhances Neutral, But Not Negative, Components of Memory.

Emotionally salient components of memory are preferentially remembered at the expense of accompanying neutral information. This emotional memory trade-off is enhanced over time, and possibly sleep, through a process of memory consolidation. Sleep is believed to benefit memory through a process of reactivation during nonrapid eye movement sleep (NREM). Here, targeted memory reactivation (TMR) was used to manipulate the reactivation of negative and neutral memories during NREM sleep. Thirty-one male and female participants encoded composite scenes containing either a negative or neutral object superimposed on an always neutral background. During NREM sleep, sounds associated with the scene object were replayed, and memory for object and background components was tested the following morning. We found that TMR during NREM sleep improved memory for neutral, but not negative scene objects. This effect was associated with sleep spindle activity, with a larger spindle response following TMR cues predicting TMR effectiveness for neutral items only. These findings therefore do not suggest a role of NREM memory reactivation in enhancing the emotional memory trade-off across a 12 h period but do align with growing evidence of spindle-mediated memory reactivation in service of neutral declarative memory.

Unsupervised Object-Centric Learning From Multiple Unspecified Viewpoints.

Visual scenes are extremely diverse, not only because there are infinite possible combinations of objects and backgrounds but also because the observations of the same scene may vary greatly with the change of viewpoints. When observing a multi-object visual scene from multiple viewpoints, humans can perceive the scene compositionally from each viewpoint while achieving the so-called "object constancy" across different viewpoints, even though the exact viewpoints are untold. This ability is essential for humans to identify the same object while moving and to learn from vision efficiently. It is intriguing to design models that have a similar ability. In this article, we consider a novel problem of learning compositional scene representations from multiple unspecified (i.e., unknown and unrelated) viewpoints without using any supervision and propose a deep generative model which separates latent representations into a viewpoint-independent part and a viewpoint-dependent part to solve this problem. During the inference, latent representations are randomly initialized and iteratively updated by integrating the information in different viewpoints with neural networks. Experiments on several specifically designed synthetic datasets have shown that the proposed method can effectively learn from multiple unspecified viewpoints.

Compositional scene modeling with global object-centric representations

Compositional scene modeling with global object-centric representations

البناء التصويري لرمزية الماء في شعر نزار قباني

Waterscenes are one of the topics employed in poetic texts, as poets were inspired in their poetic texts when they took them as a refuge for their poetic symbols, either for interpretation, or to build a cinematic and dramatic world that represents the life that the poet seeks. Nizar Qabbani is considered one of the poets who employed the subject of water in his poetry, so he made of this subject a set of symbolic connotations and suggestions by making some individual and composite images, to draw distinct vital scenes whose significance varies from one text to another, so these water images were a living theater in which the elements of the image are manifested, ranging from the individual scene to the composite scene, to meet the dramatic scene with its water components, to be vibrant with symbolic connotations. Keywords: water, construction, art, photography, single, vehicle.

Compositional Scene Representation Learning via Reconstruction: A Survey.

Visual scenes are composed of visual concepts and have the property of combinatorial explosion. An important reason for humans to efficiently learn from diverse visual scenes is the ability of compositional perception, and it is desirable for artificial intelligence to have similar abilities. Compositional scene representation learning is a task that enables such abilities. In recent years, various methods have been proposed to apply deep neural networks, which have been proven to be advantageous in representation learning, to learn compositional scene representations via reconstruction, advancing this research direction into the deep learning era. Learning via reconstruction is advantageous because it may utilize massive unlabeled data and avoid costly and laborious data annotation. In this survey, we first outline the current progress on reconstruction-based compositional scene representation learning with deep neural networks, including development history and categorizations of existing methods from the perspectives of the modeling of visual scenes and the inference of scene representations; then provide benchmarks, including an open source toolbox to reproduce the benchmark experiments, of representative methods that consider the most extensively studied problem setting and form the foundation for other methods; and finally discuss the limitations of existing methods and future directions of this research topic.

The Yeni Müzik Scene in Türkiye: How did the ‘New Music’ Discourse Change Local Contemporary Music Practice?

Although the field of contemporary music composition in Türkiye is mainly considered an institutionalization grounded on the nation-state ideology and the cultural policies of the early republic period, the political, economic, and cultural changes experienced in the last 30 years have led to the formation of alternative discourses and new institutionalizations in the field. Among these new formations, the 'yeni müzik' discourse - which can be considered as the local manifestation of the 'new music' discourse that originates in the 20th-century art music canon and the new composition scene shaped around it have marked significant differences in the local contemporary music practice. By providing a critical overview of the local history of the field, examining the early emergence of ‘yeni müzik’, and documenting the post-2000s development of this new compositional institutionalization, the present paper proposes ‘yeni müzik scene’ as an alternative formation and discusses how it differs from its predecessor 'Turkish Contemporary Music' in terms of institutional, social and musical practices. Our account of the topic – which has hardly been studied in the literature – benefits from both historiographical and fieldwork practices, hoping to provide a continuous socio-cultural narrative that situates the ‘yeni müzik scene’ within the local history of contemporary music.

State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction

Abstract3D reconstruction of deformable (or non‐rigid) scenes from a set of monocular 2D image observations is a long‐standing and actively researched area of computer vision and graphics. It is an ill‐posed inverse problem, since—without additional prior assumptions—it permits infinitely many solutions leading to accurate projection to the input 2D images. Non‐rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set‐ups such as stereo or multi‐view systems. This survey focuses on state‐of‐the‐art methods for dense non‐rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods—that handle arbitrary scenes and make only a few prior assumptions—and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high‐level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods.

Extra Consciousness, Extra Fingers: Automatic Writing and Disabled Authorship

Abstract Nineteenth-century Spiritualism championed women with chronic illnesses as the ideal conduits for mediumship due to their assumed sensitivity. Positioning the movement’s many historical iterations of automatic writing as central to disability history, this article turns to Gertrude Stein and Lucille Clifton, who center extrasensory perceptions in the compositional scene. Foregrounding mind and body, they upend the privileging of the rational male subject who dominates accounts of authorship in literary studies. By modeling collaborative forms of writing that exceed consciousness, Stein and Clifton make way for embracing disabled authorship in our past and present.

UAV multispectral multi-domain feature optimization for the air-to-ground recognition of outdoor injured human targets under cross-scene environment.

UAV-based multispectral detection and identification technology for ground injured human targets, is a novel and promising unmanned technology for public health and safety IoT applications, such as outdoor lost injured searching and battlefield casualty searching, and our previous research has demonstrated its feasibility. However, in practical applications, the searched human target always exhibits low target-background contrast relative to the vast and diverse surrounding environment, and the ground environment also shifts randomly during the UAV cruise process. These two key factors make it difficult to achieve highly robust, stable, and accurate recognition performance under the cross-scene situation. This paper proposes a cross-scene multi-domain feature joint optimization (CMFJO) for cross-scene outdoor static human target recognition. In the experiments, we first investigated the impact severity of the cross-scene problem and the necessity to solve it by designing 3 typical single-scene experiments. Experimental results show that although a single-scene model holds good recognition capability for its scenes (96.35% in desert scenes, 99.81% in woodland scenes, and 97.39% in urban scenes), its recognition performance for other scenes deteriorates sharply (below 75% overall) after scene changes. On the other hand, the proposed CMFJO method was also validated using the same cross-scene feature dataset. The recognition results for both individual scene and composite scene show that this method could achieve an average classification accuracy of 92.55% under cross-scene situation. This study first tried to construct an excellent cross-scene recognition model for the human target recognition, named CMFJO method, which is based on multispectral multi-domain feature vectors with scenario-independent, stable and efficient target recognition capability. It will significantly improve the accuracy and usability of UAV-based multispectral technology method for outdoor injured human target search in practical applications and provide a powerful supporting technology for public safety and health.

Research on SAR Imaging Simulation Based on Time-Domain Shooting and Bouncing Ray Algorithm

The signal source of synthetic aperture radar (SAR) usually adopts the linear frequency modulation (LFM) signal, which exhibits the characteristics of a wide pulse. Hence, in the case of usage of the LFM signal by time-domain shooting and bouncing ray (TDSBR) to simulate the SAR echo signal, numerous time sampling points are generated, resulting in huge computational efforts; thereby, it is hard to exploit the TDSBR algorithm for simulating the SAR echo. In order to unlock this dilemma, the hybrid approach, the transfer function in conjunction with the range frequency-domain pulse coherence, is developed, in which the transfer function is stated by the radar cross-section. The proposed methodology is capable of enhancing the computational efficiency through avoiding the massive time sampling, so that the suggested TDSBR approach could be more conveniently applied to the SAR echo simulations. Furthermore, because of the efficiency advantage of the TDSBR in the calculation of wideband scattering field, the proposed methodology exhibits higher computational efficiency than the frequency-domain shooting and bouncing ray in the SAR image simulation. Finally, the key equation of the TDSBR for the transient scattering field of dielectric targets based on a closed-form integration formula is analytically derived, which is the basis of SAR imaging simulation for the composite scene of ships above the sea surface.

ISAR Imaging for a Composite Target-Layered Rough Surface Scene Based on the Accelerated Time-Domain Scattering Algorithm

Most existing imaging studies are mainly used for a single target or a composite scene consisting of a target and a rough surface (TRS) below it, such as the target-ocean scene. This article focuses on inverse synthetic aperture radar (ISAR) imaging of a special composite scene consisting of a target and layered rough surfaces (TLRS) below it, such as a target and a snow-soil layered scene. Differently from previous imaging algorithms based on the frequency-domain (FD) method, this article combines the time-domain ray bouncing (TDRB) method with imaging algorithms to develop an accurate ISAR imaging framework to obtain composite images of the TLRS scene. To improve the simulation efficiency of the time-domain (TD) echo matrix for ISAR imaging, the multi-thread parallel is used to accelerate the TD scattering algorithm, then, the ISAR imaging formulas based on scattering field expressions in the TLRS scene are derived. Since the linear frequency modulation pulse used in traditional radar imaging has longer pulsewidth than the Gaussian, the matched filter is modified to realize pulse compression under modulating Gaussian pulse excitations to enhance the imaging efficiency. Simulation results show that the proposed ISAR imaging framework ensures the image quality and improves the simulation efficiency significantly. Finally, the ISAR images of the target-snow-soil scene with different parameters are simulated. This article addresses the correlation between the scattering mechanism and image features for ISAR images based on different scattering components, a topic that the literatures do lack.

A framework for quantitative analysis and differentiated marketing of tourism destination image based on visual content of photos

Photos shared by tourists are being generated at an unprecedented speed, creating new opportunities to study tourism destination images. Nevertheless, little research has focused on the tourist's perception of images from multiple perspectives and how to construct differentiated marketing strategies that link tourists with destinations. With the support of deep-learning technology, we propose herein a quantitative analysis strategy and differentiated marketing framework driven by photo big data which contains images of tourism destinations. We draw on photos of tourism destinations shared by tourists and analyse three aspects of perceived images – composition scene, visual aesthetic quality and visual uniqueness. We further develop a set of objective image-projection schemes that integrate multiple indicators of tourism destinations to improve destination marketing. Finally, we focus on an empirical case study of Wuyuan, China. The results of the study have methodological, theoretical and practical implications for the tourism industry.

Nonlinear Ship Wake Detection in SAR Images Based on Electromagnetic Scattering Model and YOLOv5

Traditional wake detection methods have been successfully applied to the detection of a simple linear ship wake. However, they cannot effectively detect nonlinear wake and weak wake under high sea state conditions, whereas the deep-learning-based detection method could play to its strengths in this respect. Due to the lack of sufficient measured SAR images of ship wake to meet the training requirement for deep learning method, this paper explores the method to detect the nonlinear ship wake by combining electromagnetic scattering model with deep learning technique. The composite scene model of the sea surface and its wake is established first, then the facet scattering distribution of the ship wake and the sea background is analyzed with the help of the electromagnetic scattering model, and the simulation of the wake SAR images under the sea background is finally accomplished based on the modulation model. Combined with the simulation results and measured wake SAR images, the sample database is constructed. The You Only Look Once Version five algorithm (YOLOv5) based on deep learning techniques is applied to detect the wake target in complex conditions such as different sea states, multiple targets, curvilinear wakes, and weak wakes. The result show that the YOLOv5 leads to an obvious higher detection efficiency with satisfactory accuracy. Moreover, the comparison between YOLOv5 and the traditional Radon transform method for detecting nonlinear wakes in a strong noise background shows that the proposed method is better than the traditional object detection model. Thus, the proposed scheme would be a practical tool to deal with the detection of nonlinear ship wake and weak wake in complex scenarios, which will be helpful to the further remote sensing investigation of the ship.

A Hybrid Parametric Scattering Model for an Object on a Dielectric Rough Surface

In this communication, we combine the facet-based bidirectional reflection distribution function (BRDF) and four-path model (FPM) to establish the attribute scattering center (ASC) model for composite scattering simulations. This hybrid method is beneficial for simulating scattering echoes of large composite scenes with high efficiency. The model consists of three parts: the ASC model for the target, the BRDF-ASC model for the rough dielectric background, and the FPM-BRDF-ASC model for the coupling scattering between the target and the background. Two simple composite scenes and two large composite scenes are taken as examples to verify the accuracy and feasibility of the proposed model. The validation results show that the radar cross sections (RCSs) of the proposed hybrid method are consistent with those of the multilevel fast multipole algorithm (MLFMA) with higher simulation efficiency. In addition, the ghosting image caused by the multipath effect can be well displayed in the simulated synthetic aperture radar (SAR) images.

Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpoints

Visual scenes are extremely rich in diversity, not only because there are infinite combinations of objects and background, but also because the observations of the same scene may vary greatly with the change of viewpoints. When observing a visual scene that contains multiple objects from multiple viewpoints, humans are able to perceive the scene in a compositional way from each viewpoint, while achieving the so-called ``object constancy'' across different viewpoints, even though the exact viewpoints are untold. This ability is essential for humans to identify the same object while moving and to learn from vision efficiently. It is intriguing to design models that have the similar ability. In this paper, we consider a novel problem of learning compositional scene representations from multiple unspecified viewpoints without using any supervision, and propose a deep generative model which separates latent representations into a viewpoint-independent part and a viewpoint-dependent part to solve this problem. To infer latent representations, the information contained in different viewpoints is iteratively integrated by neural networks. Experiments on several specifically designed synthetic datasets have shown that the proposed method is able to effectively learn from multiple unspecified viewpoints.

Transient Scattering Echo Simulation and ISAR Imaging for a Composite Target-Ocean Scene Based on the TDSBR Method

We propose an inverse synthetic aperture radar (ISAR) imaging algorithm for a composite target-ocean scene based on time-domain shooting and bouncing rays (TDSBR) method. To do this, we develop the TDSBR method to simulate the electromagnetic scattering echoes from a composite target-ocean scene. Then, we derive the ISAR imaging formulas based on electromagnetic scattering field expressions. The demodulation, pulse compression, and azimuth inverse fast Fourier transform are performed on the scattering echo data to obtain ISAR images. Linear frequency modulation (LFM) pulses are usually used in traditional radar imaging. However, its long pulse width leads to more computation time for simulations. To increase efficiency, we replace LFM pulses with modulating Gaussian pulses. As a result, we modify the matched filter to achieve pulse compression under modulating Gaussian pulse excitations. Simulation results confirm that this proposed method improves efficiency while ensuring image quality. This proposed technique fully incorporates the electromagnetic scattering mechanism and is accurate. It effectively combines the electromagnetic scattering algorithm and the radar imaging algorithm. Finally, we present ISAR images simulated by this method. These results confirm the feasibility and efficiency of this technique.

Infrared and visible image fusion based on visibility enhancement and norm optimization low-rank representation

Infrared and visible image fusion aims to fuse infrared targets and visible details in a composite scene. Recently, many fusion methods have been proposed, but most cannot balance characteristics of infrared and visible images well. We propose an infrared and visible image fusion method. First, to enhance the quality of the infrared and visible images to assist the subsequent prefusion and decomposition, a preprocessing method based on singular value decomposition combined with high dynamic range compression and guided filter (GF) is proposed. Then the preprocessing results are fused to produce the prefusion result by multiscale structural image decomposition-based fusion method. Next, the prefusion result is decomposed by the multilevel image decomposition method based on latent low-rank representation (MDLatLRR). Then the source images are decomposed by MDLatLRR. For the base layer, we combine the visual saliency map with GF to obtain the final fused base layer. For the detail layers, we propose a nonlinear function to highlight the infrared target and use weighted least square to optimize the details. We also adopt a structural similarity-based weight map via L2-norm to improve the fusion performance. The experimental results demonstrate that the proposed method outperforms some state-of-the-art methods both qualitatively and quantitatively.

ISAR Imaging for Target Above Rough Surface Based on Time-Domain Scattering Echo

The inverse synthetic aperture radar (ISAR) imaging of the target above the rough surface based on the time-domain (TD) electromagnetic (EM) scattering algorithms is focused on in this paper. It effectively combines TD ray tracing (TDRT) method for the composite scene and radar imaging model. The scattering echoes for ISAR imaging are solved by TDRT that the TD mutual coupling between the target and rough surface can be considered by the ray bounce strategy. Compared with the frequency domain ray tracing (FDRT) and multilevel fast multi-pole method (MFMM), TDRT algorithm not only ensures the calculation accuracy, but also greatly reduces the calculation cost. Then, we generate the ISAR image of the target above the rough surface according to the echo matrix based on TDRT. In order to improve the imaging quality of the target, the non-linear mapping theory is presented to suppress the virtual image points and noisy points and it is beneficial to the future research of target recognition in the composite scene.

‘Our “China Within”’: Minjian Sinopsy Today, a Segue

In this article I have set out to draw an ethnographically inflected, composite scene of what I would loosely term ‘Sinopsy’ today, drawing on a series of explorative conversations on ‘psychoanalysis in China’ and the questionnaire-based interviews I undertook (between 2015 and 2020) with 18 psychoanalysts, psychotherapists, psychological counselors and engaged academics in or from mainland China, ranging from seasoned professionals to new-generation trainees. My ongoing, modest hope is to get to see a bigger and ‘democalligraphically’ evolving picture of a kind of praxis-oriented community-serving minjian psychoanalysis on the ground. As I turn to this understudied, cartographically complex, porously open-ended zone of Sinopsychoanalysis in the making, a silhouette seems to be emerging on the horizon, itself a question in motion: What is (in) it for people in China, and across and beyond its great walls? Focusing on its transitional specificity, its active indeterminacy and eclectic adaptivity exemplified by Sino-Lacanian analysts’ practices among others, I also try to contextualize its deeper and broader psychocultural dimensions, especially given the turbulent (post)modernity of China, where its ongoing epochal traumas are inextricably private and public, familial, national and diasporic.