Scene Composition Research Articles

CineMPC: A Fully Autonomous Drone Cinematography System Incorporating Zoom, Focus, Pose, and Scene Composition

CineMPC: A Fully Autonomous Drone Cinematography System Incorporating Zoom, Focus, Pose, and Scene Composition

Generating novel scene compositions from single images and videos

Given a large dataset for training, generative adversarial networks (GANs) can achieve remarkable performance for the image synthesis task. However, training GANs in extremely low data regimes remains a challenge as overfitting often occurs, leading to memorization or training divergence. In this work, we introduce SIV-GAN, an unconditional generative model that can generate new scene compositions from a single training image or a single video clip. We propose a two-branch discriminator architecture with content and layout branches that are designed to judge internal content and scene layout realism separately from each other. This discriminator design enables the synthesis of visually plausible, novel compositions of a scene with varying content and layout while preserving the context of the original sample. Compared to previous single-image GANs, our model generates more diverse images of higher quality while not being restricted to a single image setting. We further introduce a new challenging task of learning from a few frames of a single video. In this training setup the training images are highly similar to each other, which makes it difficult for prior GAN models to achieve a synthesis of both high quality and diversity.

A speaking silence: “universal language” and multilingualism in The Shape of Water

Framed by Guillermo del Toro as “a love letter to the cinema,” the academy award-winning feature The Shape of Water (2017) speaks to one of the core debates in film studies: film’s status as a “universal language.” Paradoxically both propagating and critiquing this idea of universality, a vast knowledge of film history, cinematic techniques, and a desire to create what the director calls a “cinematic Esperanto”—or a globally legible film tradition—informs a film that celebrates the multiplicity of “languages” possible in film as a medium. Reading The Shape of Water as a series of films-within-films—fantasy within realism, black-and-white within color, and a silent film within a “talkie”—I reveal the myriad communicative forms accessible within cinema as a visual, temporal, and mobile medium. The foregrounding of “a speaking silence” in the interactions between the mute Elisa and the Amphibian Man, a pre-linguistic, but communicative, sea creature, acts as a set piece—a silent film within the larger film proper—that celebrates the universal language of gesture, while the situation of this nested narrative within a larger film tradition of “talkies” speaks to the ultimate displacement of the fantasy of universality by the adoption of the voice in film. Through a close attention to film stills, the formal composition of scenes, and the sparse, multilingual dialogue, I examine The Shape of Water’s critique of a monolingual “universal” filmic language and celebration of the radical multilingualism possible in the cinema.

Punk and decolonial thinking in Bogota, Colombia

This article gathers the work carried out with three punk bands of Bogota to talk about decoloniality regarding race, class and gender. To analyse the latter the theoretical framework we proposed was decolonial aesthetics and liberating music praxis. The results from the research were two-fold: on the one hand, through a quantitative methodology, Bogota’s punk scene composition regarding race, class and gender; and on the other hand, through a qualitative methodology, a discursive analysis of the workshops, the in-depth interviews and the lyrics of the songs. We sought to answer the question: how does punk connect with decoloniality and become a music genre that supports the narration of oppressions?

Adaptive Discriminative Regions Learning Network for Remote Sensing Scene Classification.

As an auxiliary means of remote sensing (RS) intelligent interpretation, remote sensing scene classification (RSSC) attracts considerable attention and its performance has been improved significantly by the popular deep convolutional neural networks (DCNNs). However, there are still several challenges that hinder the practical applications of RSSC, such as complex composition of land cover, scale-variation of objects, and redundant and noisy areas for scene classification. In order to mitigate the impact of these issues, we propose an adaptive discriminative regions learning network for RSSC, referred as ADRL-Net briefly, which locates discriminative regions effectively for boosting the performance of RSSC by utilizing a novel self-supervision mechanism. Our proposed ADRL-Net consists of three main modules, including a discriminative region generator, a region discriminator, and a region scorer. Specifically, the discriminative region generator first generates some candidate regions which could be informative for RSSC. Then, the region discriminator evaluates the regions generated by region generator and provides feedback for the generator to update the informative regions. Finally, the region scorer makes prediction scores for the whole image by using the discriminative regions. In such a manner, the three modules of ADRL-Net can cooperate with each other and focus on the most informative regions of an image and reduce the interference of redundant regions for final classification, which is robust to the complex scene composition, object scales, and irrelevant information. In order to validate the efficacy of the proposed network, we conduct experiments on four widely used benchmark datasets, and the experimental results demonstrate that ADRL-Net consistently outperforms other state-of-the-art RSSC methods.

Listen to ‘Mila’, Listen to the Hong Kong's Social Soundscape on the Contemporary Opera Stage (Review)

The creation and performance of contemporary opera in the 21st century reflect the main ideas of contemporary humanistic trend of thought. Contemporary opera is increasingly deepening in international cooperation, cross- cultural artistic expression, global exploration of local social issues, and artists' social participation. Today is a new humanistic era for contemporary opera and other art forms. The artistic and humanistic languages of different nationalities and cultures communicate and dialogue on more diverse artistic platforms. Therefore, the creation of realistic opera which reflects the diversity of Chinese humanistic values and the real development of Chinese society is particularly important today. The creation and performance of contemporary opera in Hong Kong is also a representative of the development of Chinese opera with remarkable regional characteristics. Taking ‘Mila’’, an opera commissioned by the Asia Society Hong Kong Center, as an example, this article analyzes the realistic librettural themes, the music composition of contemporary scenes, and the dramatic presentation of life materials in the work, so as to discover Hong Kong's social sound landscape on the contemporary opera stage.

<그것이 알고 싶다>의 범죄 사건 스토리텔링 전략 연구

Today, real criminal cases have entered a phase, wherein they are disseminated as socio-cultural content. The purpose of this article is to examine the mass media’s content production strategy for sharing and consuming criminal cases. This article focuses on the strategies by analyzing the 1314th episode of “The Raincoat Killer on Stage” and focusing on the characteristics of different platforms such as TV and YouTube, as well as their correlations with each other. In the episode of “Raincoat Killer on Stage,”a third party (character), distinguished from the host, appeared and positioned the detective’s gaze on the mystery toward the middle of the episode. In addition, by appropriating the art style of “play” as a storytelling method, the program’s unique dramatization strategy is considerably emphasized at various layers of story structure and scene composition. This is in line with the idea of coping with changes in the media environment, by satisfying the tastes of the viewership who are enthusiastic about crime mysteries, to maintain its influence and use it as the basis for the social function of the investigation program. While the production crew’s performativity is highly emphasized in the TV episode of “The Unanswered,”the fact that the intimate conversational relationship between the production subject and audience is fully broadened on the YouTube channel confirms this media strategy. The investigation program’s duplicity to obtain the capacity of agenda setting or agenda keeping should be reconsidered, particularly as its genre nature of non-fiction is internally collapsed, presuming the image of the media populace consuming real criminal cases for entertainment.

Scene-selective brain regions respond to embedded objects of a scene.

Objects are fundamental to scene understanding. Scenes are defined by embedded objects and how we interact with them. Paradoxically, scene processing in the brain is typically discussed in contrast to object processing. Using the BOLD5000 dataset (Chang etal., 2019), we examined whether objects within a scene predicted the neural representation of scenes, as measured by functional magnetic resonance imaging in humans. Stimuli included 1,179 unique scenes across 18 semantic categories. Object composition of scenes were compared across scene exemplars in different semantic scene categories, and separately, in exemplars of the same scene category. Neural representations in scene- and object-preferring brain regions were significantly related to which objects were in a scene, with the effect at times stronger in the scene-preferring regions. The object model accounted for more variance when comparing scenes within the same semantic category to scenes from different categories. Here, we demonstrate the function of scene-preferring regions includes the processing of objects. This suggests visual processing regions may be better characterized by the processes, which are engaged when interacting with the stimulus kind, such as processing groups of objects in scenes, or processing a single object in our foreground, rather than the stimulus kind itself.

VR4EPCONSULT: REMOTE PROCEDURE CONSULTATION FOR ELECTROPHYSIOLOGY PROCEDURES USING VIRTUAL REALITY

BackgroundInvasive catheter-based approaches for challenging and complex cardiac arrhythmias requires expertise which is not readily available across various medical centers around the world. Due to overall limitation of expert proceduralists there can be suboptimal procedural outcome, avoidable procedural complications, re-do procedures, and a rise in cost across patient, payer, and healthcare utilization resources. Virtual Reality (VR) can minimize the above-mentioned concerns and facilitate anytime, anywhere procedure consultation.ObjectiveWe hypothesize that a Remote Procedural Consultant (RPC) can enhance patient safety, procedural success, and proceduralist satisfaction. In partnership with AirV Labs, we developed a VR4EPConsult framework that includes bi-directional communication and interactions to facilitate real-time procedural assistance between clinical sites.MethodsDeveloped using Unity3D™, and a proprietary collaboration platform AirVu™, VR4EPConsult (Figures 1 & 2) consists of the following modules to facilitate RPC between host and consultant sites: (i) synchronization to co-ordinate multi-modal interactions (audio, images, annotations, 3D models, live feeds of ultrasound, fluoroscopy, mapping system), (ii) visualization (scene composition including 360 live video, camera placement, rendering of modalities), and (iii) exploration (teleport, positioning and manipulation of multiple modalities) to strategize Electrophysiology (EP) procedure.ResultsVR4EPConsult proof-of-concept offers a real-time, interactive, procedural consultation anytime, anywhere by connecting a host with expert proceduralists. We report a case of complex ablation procedure which was analyzed and ablated using VR4EPConsult.ConclusionFigure 2: Snapshot of VR4EPConsult using Multiple modalities for Remote Procedural ConsultantView Large Image Figure ViewerDownload Hi-res image Download (PPT) BackgroundInvasive catheter-based approaches for challenging and complex cardiac arrhythmias requires expertise which is not readily available across various medical centers around the world. Due to overall limitation of expert proceduralists there can be suboptimal procedural outcome, avoidable procedural complications, re-do procedures, and a rise in cost across patient, payer, and healthcare utilization resources. Virtual Reality (VR) can minimize the above-mentioned concerns and facilitate anytime, anywhere procedure consultation. Invasive catheter-based approaches for challenging and complex cardiac arrhythmias requires expertise which is not readily available across various medical centers around the world. Due to overall limitation of expert proceduralists there can be suboptimal procedural outcome, avoidable procedural complications, re-do procedures, and a rise in cost across patient, payer, and healthcare utilization resources. Virtual Reality (VR) can minimize the above-mentioned concerns and facilitate anytime, anywhere procedure consultation. ObjectiveWe hypothesize that a Remote Procedural Consultant (RPC) can enhance patient safety, procedural success, and proceduralist satisfaction. In partnership with AirV Labs, we developed a VR4EPConsult framework that includes bi-directional communication and interactions to facilitate real-time procedural assistance between clinical sites. We hypothesize that a Remote Procedural Consultant (RPC) can enhance patient safety, procedural success, and proceduralist satisfaction. In partnership with AirV Labs, we developed a VR4EPConsult framework that includes bi-directional communication and interactions to facilitate real-time procedural assistance between clinical sites. MethodsDeveloped using Unity3D™, and a proprietary collaboration platform AirVu™, VR4EPConsult (Figures 1 & 2) consists of the following modules to facilitate RPC between host and consultant sites: (i) synchronization to co-ordinate multi-modal interactions (audio, images, annotations, 3D models, live feeds of ultrasound, fluoroscopy, mapping system), (ii) visualization (scene composition including 360 live video, camera placement, rendering of modalities), and (iii) exploration (teleport, positioning and manipulation of multiple modalities) to strategize Electrophysiology (EP) procedure. Developed using Unity3D™, and a proprietary collaboration platform AirVu™, VR4EPConsult (Figures 1 & 2) consists of the following modules to facilitate RPC between host and consultant sites: (i) synchronization to co-ordinate multi-modal interactions (audio, images, annotations, 3D models, live feeds of ultrasound, fluoroscopy, mapping system), (ii) visualization (scene composition including 360 live video, camera placement, rendering of modalities), and (iii) exploration (teleport, positioning and manipulation of multiple modalities) to strategize Electrophysiology (EP) procedure. ResultsVR4EPConsult proof-of-concept offers a real-time, interactive, procedural consultation anytime, anywhere by connecting a host with expert proceduralists. We report a case of complex ablation procedure which was analyzed and ablated using VR4EPConsult. VR4EPConsult proof-of-concept offers a real-time, interactive, procedural consultation anytime, anywhere by connecting a host with expert proceduralists. We report a case of complex ablation procedure which was analyzed and ablated using VR4EPConsult. Conclusion

Quad‐fisheye Image Stitching for Monoscopic Panorama Reconstruction

AbstractMonoscopic panorama provides the display of omnidirectional contents surrounding the viewer. An increasingly popular way to reconstruct a panorama is to stitch a collection of fisheye images. However, such non‐planar views may result in problems such as distortions and boundary irregularities. In most cases, the computational expense for stitching non‐planar images is also too high to satisfy real‐time applications. In this paper, a novel monoscopic panorama reconstruction pipeline that produces better quad‐fisheye image stitching results for omnidirectional environment viewing is proposed. The main idea is to apply mesh deformation for image alignment. To optimize inter‐lens parallaxes, unwarped images are firstly cropped and reshuffled to facilitate the circular environment scene composition by the seamless ring‐connection of the panorama borders. Several mesh constraints are then adopted to ensure a high alignment accuracy. After alignment, the boundary of the result is rectified to be rectangular to prevent gapping artefacts. We further extend our approach to video stitching. The temporal smoothness model is added to prevent unexpected artefacts in the panoramic videos. To support interactive applications, our stitching algorithm is programmed using CUDA. The camera motion and average gradient per video frame are further calculated to accelerate for synchronous real‐life panoramic scene reconstruction and visualization. Experimental results demonstrate that our method has advantages in respects of alignment accuracy, adaptability and image quality of the stitching result.

The construction of split whiteness in the queer films Kanarie (2018) and Moffie (2019)

After the end of formal apartheid, a number of South African feature films have explored queer white men in conservative social settings, with a particular focus on Afrikaans-speaking gay men. These films have reflected strict heteropatriarchal values within white Afrikaner culture where homosexuality is still often seen as a taboo topic. In this article I discuss two feature films with gay white male protagonists, Kanarie (2018) by Christiaan Olwagen and Moffie (2019) by Oliver Hermanus. These films both feature young men conscripted to fight in the South African Border War in the 1980s, but differ greatly in terms of genre, plot, and style. I argue that, while many scholars discuss whiteness as a general construct that affords privilege, the films demonstrate a split whiteness that is effected through the composition of particular shots and scenes as well as through the films’ processes of production and reception. Whiteness is split into an invisibilised, assumedly critical perspective on the one hand, with transnational links to the Global North, and a hypervisibilised, reified, and criticised racial identity on the other hand, located specifically in the heteropatriarchal Afrikaner male. Queer characters in both films are able to split their identities and dissociate from uncomfortable parts of their whiteness, taking on an assumed criticality that highlights their own oppression and exclusion. The films thus dismiss the protagonists’ complicity in white supremacy, and allow audiences to dissociate from their own complicity in anti-black violence and oppression.

Aesthetic-guided outward image cropping

Image cropping is a commonly used post-processing operation for adjusting the scene composition of an input photography, therefore improving its aesthetics. Existing automatic image cropping methods are all bounded by the image border, thus have very limited freedom for aesthetics improvement if the original scene composition is far from ideal, e.g. the main object is too close to the image border. In this paper, we propose a novel, aesthetic-guided outward image cropping method. It can go beyond the image border to create a desirable composition that is unachievable using previous cropping methods. Our method first evaluates the input image to determine how much the content of the image should be extrapolated by a field of view (FOV) evaluation model. We then synthesize the image content in the extrapolated region, and seek an optimal aesthetic crop within the expanded FOV, by jointly considering the aesthetics of the cropped view, and the local image quality of the extrapolated image content. Experimental results show that our method can generate more visually pleasing image composition in cases that are difficult for previous image cropping tools due to the border constraint, and can also automatically degrade to an inward method when high quality image extrapolation is infeasible.

Write‐An‐Animation: High‐level Text‐based Animation Editing with Character‐Scene Interaction

Abstract3D animation production for storytelling requires essential manual processes of virtual scene composition, character creation, and motion editing, etc. Although professional artists can favorably create 3D animations using software, it remains a complex and challenging task for novice users to handle and learn such tools for content creation. In this paper, we present Write‐An‐Animation, a 3D animation system that allows novice users to create, edit, preview, and render animations, all through text editing. Based on the input texts describing virtual scenes and human motions in natural languages, our system first parses the texts as semantic scene graphs, then retrieves 3D object models for virtual scene composition and motion clips for character animation. Character motion is synthesized with the combination of generative locomotions using neural state machine as well as template action motions retrieved from the dataset. Moreover, to make the virtual scene layout compatible with character motion, we propose an iterative scene layout and character motion optimization algorithm that jointly considers character‐object collision and interaction. We demonstrate the effectiveness of our system with customized texts and public film scripts. Experimental results indicate that our system can generate satisfactory animations from texts.

Emotional Responses to the Visual Patterns of Urban Streets: Evidence from Physiological and Subjective Indicators.

Despite recent progress in the research of people’s emotional response to the environment, the built—rather than natural—environment’s emotional effects have not yet been thoroughly examined. In response to this knowledge gap, we recruited 26 participants and scrutinized their emotional response to various urban street scenes through an immersive exposure experiment using virtual reality. We utilized new physiological monitoring technologies that enable synchronized observation of the participants’ electroencephalography, electrodermal activity, and heart rate, as well as their subjective indicators. With the newly introduced measurement for the global visual patterns of the built environment, we built statistical models to examine people’s emotional response to the physical element configuration and color composition of street scenes. We found that more diverse and less fragmented scenes inspired positive emotional feelings. We also found (in)consistency among the physiological and subjective indicators, indicating a potentially interesting neural−physiological interpretation for the classic form−function dichotomy in architecture. Besides the practical implications on promoting physical environment design, this study combined objective physiology-monitoring technology and questionnaire-based research techniques to demonstrate a better approach to quantify environment−emotion relationships.

Shortwave Radiance to Irradiance Conversion for Earth Radiation Budget Satellite Observations: A Review

Observing the Earth radiation budget (ERB) from satellites is crucial for monitoring and understanding Earth’s climate. One of the major challenges for ERB observations, particularly for reflected shortwave radiation, is the conversion of the measured radiance to the more energetically relevant quantity of radiative flux, or irradiance. This conversion depends on the solar-viewing geometry and the scene composition associated with each instantaneous observation. We first outline the theoretical basis for algorithms to convert shortwave radiance to irradiance, most commonly known as empirical angular distribution models (ADMs). We then review the progression from early ERB satellite observations that applied relatively simple ADMs, to current ERB satellite observations that apply highly sophisticated ADMs. A notable development is the dramatic increase in the number of scene types, made possible by both the extended observational record and the enhanced scene information now available from collocated imager information. Compared with their predecessors, current shortwave ADMs result in a more consistent average albedo as a function of viewing zenith angle and lead to more accurate instantaneous and mean regional irradiance estimates. One implication of the increased complexity is that the algorithms may not be directly applicable to observations with insufficient accompanying imager information, or for existing or new satellite instruments where detailed scene information is not available. Recent advances that complement and build on the base of current approaches, including machine learning applications and semi-physical calculations, are highlighted.

Exture Oriented Scene Generation from Natural Language Text Description using 3D Coloured Objects

To understand language, we need an understanding of the world around us. Language describes the world and provides symbols with which we represent meaning. Still, much knowledge about the world is so obvious that it is rarely explicitly stated. It is uncommon for people to state that chairs are usually on the floor and upright, and that you usually eat a cake from a plate on a table. Knowledge of such common facts provides the context within which people communicate with language. Therefore, to create practical systems that can interact with the world and communicate with people, we need to leverage such knowledge to interpret language in context. Scene generation can be used to achieve an ability to generate 3D scenes on basis of text description. A model capable of learning natural language semantics or interesting pattern to generate abstract idea behind scene composition is interesting [1].Scene generation from text involves several fields like NLP, artificial intelligence, computer vision and machine learning. This paper focuses on optimally arranging objects in a room with focus on the orientation of the objects with respect to the floor, wall and ceiling of a room along with textures. Our model suggest a novel framework which can be used as a tool to generate scene where anyone without 3D Modeling.

Fit for the Job: Proportion and the Portrayal of Cattle in Egyptian Old and Middle Kingdom Elite Tomb Imagery

Depictions of the natural world are an intrinsic feature of Egyptian visual culture, with the vast array of imagery documenting animals a testimony to the fundamental role they played. Despite the significance of animals in Egypt, an anthropocentric bias still exists in research on the methods used by practitioners during initial scene composition. To help bridge the divide, the author herein undertook an investigation to determine if proportional guides were in place when rendering animal figures in ancient Egyptian elite tomb imagery of the Old and Middle Kingdoms. A notable outcome of the proportional analysis was the identification of two distinct body-types for domestic cattle (Bos taurus taurus). The aim of the current paper is to further examine these proportional differences to explore if variations in physique (namely the distance between the chest floor and withers) were rendered by Egyptian practitioners to reflect the conditions in which they appeared by considering two overarching factors: (1) biological factors and (2) contextual factors. As such, the study will employ proportional analysis to challenge the prevailing perspective of a deregulated approach when illustrating fauna in elite tomb imagery, highlighting the significance of animals within ancient Egypt.

Roman Model-Books as a Resource for Digital Architectural Reconstructions

Upon examination of Roman landscape paintings preserved in situ and in museums of Naples and Rome, additional evidence has been found for the additive character of creation of imaginary landscapes as well as evidence for using standardized elements and whole scene compositions in Roman painting. This attitude is compared to the modern way of creating virtual landscapes—computer game level design and the process called “kitbashing”. I propose that both these processes share the same task to create a familiar landscape using a visual language understandable to its contemporary viewer, but also a very similar method of using predefined elements.

Learning Perceptual Aesthetics of 3-D Shapes From Multiple Views.

The quantification of 3-D shape aesthetics has so far focused on specific shape features and manually defined criteria such as the curvature and the rule of thirds. In this article, we built a model of 3-D shape aesthetics directly from human aesthetics preference data and show it to be well aligned with human perception of aesthetics. To build this model, we first crowdsource a large number of human aesthetics preferences by showing shapes in pairs in an online study and then use the same to build a 3-D shape multiview-based deep neural network architecture to allow us to learn a measure of 3-D shape aesthetics. In comparison to previous approaches, we do not use any predefined notions of aesthetics to build our model. Our algorithmically computed measure of shape aesthetics is beneficial to a range of applications in graphics such as search, visualization, and scene composition.

Lógica interna de los “crescendos” en los números de los espectáculos de Cirque du Soleil (1986-2005)

El objetivo es analizar cómo la compañía de circo contemporáneo Cirque du Soleil aumenta progresivamente la intensidad dramática de las escenas en sus creaciones para poder trasladar su forma de composición a otras creaciones, y asimismo llegar a proponer los criterios utilizados, en un contexto pedagógico que utilice el circo como recurso educativo. La metodología utilizada es la observacional, mediante la aplicación de un formato de campo, aplicado al estudio de los diez espectáculos de la compañía estrenados entre 1986-2005. Este instrumento se realizó aplicando las lentes de la ciência de la acción motriz, la praxiología motriz. Resultados y discusion: Los resultados muestran diversas formas de acentuación de la intensidad dramática, vinculadas a los componentes de la lógica interna del espectáculo circense: a los compañeros y compañeras, aumentando el número de personas implicadas en la realización de acciones motrices; al espacio, variando las figuras, aumentando la dificultad y riesgo de las mismas, elevando la altura en la que se realizan, o disminuyendo la superficie de apoyo; al tiempo, prolongando los tiempos de vuelo, manteniendo el tiempo de ejecución de diversas figuras, o incrementando la velocidad de ejecución; y a los objetos, aumentando el número de objetos implicados en los números. Las conclusiones señalan que el conocimiento exhaustivo de los componentes de la lógica interna del circo permitirá recrear crescendos en la dramaturgia de los números de circo. Como consideraciones finales indicar que estas formas de crescendo se utilizan en la progresión de los números de una compañía profesional, pero son igualmente aplicables a la composición de escenas de circo en un contexto educativo.