Sequence Of Scenes Research Articles

SSAGCN: Social Soft Attention Graph Convolution Network for Pedestrian Trajectory Prediction.

Pedestrian trajectory prediction is an important technique of autonomous driving. In order to accurately predict the reasonable future trajectory of pedestrians, it is inevitable to consider social interactions among pedestrians and the influence of surrounding scene simultaneously, which can fully represent the complex behavior information and ensure the rationality of predicted trajectories obeyed realistic rules. In this article, we propose one new prediction model named social soft attention graph convolution network (SSAGCN), which aims to simultaneously handle social interactions among pedestrians and scene interactions between pedestrians and environments. In detail, when modeling social interaction, we propose a new social soft attention function, which fully considers various interaction factors among pedestrians. Also, it can distinguish the influence of pedestrians around the agent based on different factors under various situations. For the scene interaction, we propose one new sequential scene sharing mechanism. The influence of the scene on one agent at each moment can be shared with other neighbors through social soft attention; therefore, the influence of the scene is expanded both in spatial and temporal dimensions. With the help of these improvements, we successfully obtain socially and physically acceptable predicted trajectories. The experiments on public available datasets prove the effectiveness of SSAGCN and have achieved state-of-the-art results. The project code is available at https://github.com/WW-Tong/ssagcn_for_path_prediction.

Abstract thematic roles in infants’ representation of social events

Infants' thoughts are classically characterized as iconic, perceptual-like representations.1,2,3 Less clear is whether preverbal infants also possess a propositional language of thought, where mental symbols are combined according to syntactic rules, very much like words in sentences.4,5,6,7,8,9,10,11,12,13,14,15,16,17 Because it is rich, productive, and abstract, a language of thought would provide a key to explaining impressive achievements in early infancy, from logical inference to representation of false beliefs.18,19,20,21,22,23,24,25,26,27,28,29,30,31 A propositional language-including a language of thought5-implies thematic roles that, in a sentence, indicate the relation between noun and verb phrases, defining who acts on whom; i.e., who is the agent and who is the patient.32,33,34,35,36,37,38,39 Agent and patient roles are abstract in that they generally apply to different situations: whether A kicks, helps, or kisses B, A is the agent and B is the patient. Do preverbal infants represent abstract agent and patient roles? We presented 7-month-olds (n= 143) with sequences of scenes where the posture or relative positioning of two individuals indicated that, across different interactions, A acted on B. Results from habituation (experiment 1) and pupillometry paradigms (experiments 2 and 3) demonstrated that infants showed surprise when roles eventually switched (B acted on A). Thus, while encoding social interactions, infants fill in an abstract relational structure that marks the roles of agent and patient and that can be accessed via different event scenes and properties of the event participants (body postures or positioning). This mental process implies a combinatorial capacity that lays the foundations for productivity and compositionality in language and cognition.

Sequential selection and calibration of video frames for 3D outdoor scene reconstruction

Abstract3D scene understanding and reconstruction aims to obtain a concise scene representation from images and reconstruct the complete scene, including the scene layout, objects bounding boxes and shapes. Existing holistic scene understanding methods primarily recover scenes from single images, with a focus on indoor scenes. Due to the complexity of real‐world, the information provided by a single image is limited, resulting in issues such as object occlusion and omission. Furthermore, captured data from outdoor scenes exhibits characteristics of sparsity, strong temporal dependencies and a lack of annotations. Consequently, the task of understanding and reconstructing outdoor scenes is highly challenging. The authors propose a sparse multi‐view images‐based 3D scene reconstruction framework (SMSR). It divides the scene reconstruction task into three stages: initial prediction, refinement, and fusion stage. The first two stages extract 3D scene representations from each viewpoint, while the final stage involves selection, calibration and fusion of object positions and orientations across different viewpoints. SMSR effectively address the issue of object omission by utilizing small‐scale sequential scene information. Experimental results on the general outdoor scene dataset UrbanScene3D‐Art Sci and our proprietary dataset Software College Aerial Time‐series Images, demonstrate that SMSR achieves superior performance in the scene understanding and reconstruction.

Blind video quality assessment based on Spatio-Temporal Feature Resolver

As a method to measure video quality, blind video quality assessment (BVQA) plays an important role in video related applications. Feature extraction, as a vital component of BVQA, significantly impacts the performance and speed of the method. In the process of designing BVQA method, we make the feature extraction process independent of the whole quality evaluation method. First, a Spatio-Temporal Feature Resolver (STFR) is obtained by training. Then, the STFR is employed to directly extract the spatio-temporal features of the video sequences. Finally, the extracted features are mapped to quality scores using Support Vector Regression (SVR). STFR only needs to be trained in an undistorted video sequence, and can be directly applied to video sequences of various scenes, which has a universality. To evaluate the effectiveness of the proposed method, the experimental results on four published video quality databases show that the proposed BVQA method not only achieves more accurately than the existing BVQA methods, but also exhibits significant competitiveness in computational speed.

Stabilization of turbulence-degraded video using patch-based reference frame

Imaging over long distances in the atmosphere can result in significant degradation of the acquired videos due to atmospheric turbulence. This degradation can include problems such as blurring and geometric distortion, and it is challenging to correct these distortions because of their randomness in nature. To address this, a new method for improving the geometrical quality of video sequences of remote stationary scenes is introduced in this paper. The method uses a patch-based approach to obtain a reference frame from the distorted video sequence, selecting the best quality patches from different frames. A window-based image registration method is then used to estimate the geometrical shifts of the pixels, which are used to restore a high-quality frame. The proposed method is compared with two similar state-of-the-art video stabilization methods using experiments on both synthetic and real video sequences. Results demonstrate that the patch-based method outperforms the other methods in terms of accuracy.

Комедия В. П. Катаева «Время, вперед!»: художественный образ главного героя в контексте вариантов произведения

The article studies the variants of Valentin Kataev’s comedy Time, forward! in order to identify edits that influenced the author’s expression and the reader’s perception of the image of the main character. While researching comedy variants, it was found that the authors interested in Kataev’s work avoided the comedy Time, forward!. In our opinion, knowing about the corrections of the text of the play, Soviet critics, guided by political correctness, did not dare to engage in public polemics and literary-critical discussion around the famous writer. In the post-Soviet period, Kataev’s dramaturgy lost its relevance, did not kindle the interest of specialists. This study was conducted on the material of the comedy editions published during the author’s lifetime. Its relevance is justified by establishing the originality of the image of the main character of the main version of the work, reflecting the characters, emotions and feelings of people of its era. As part of the sample analysis it was found that by making corrections to the climactic scenes of the ending of the 1934 play, the author leaves the plot, the sequence of scenes, the system of images, the general idea of the work unchanged. During the comparison of the variants of the work, the edits made by the author to the two scenes of the finale were revealed. These scenes, connecting with the preceding scene, create a single plot and ideological and emotional node of the climax of the work. The image of the characters and the hero acquire a completely different appearance. With the appearance of comedy in public and up to the present, it has been believed that the image of the main character is a passionate and uncompromising fighter for the communist attitude to work. The edits revealed in the study in the version of the comedy of 1934, without dismissing the hero’s struggle for the result, a high pace in the socialist competition, indicate self-assertion of personality: high morality, friendly feelings. The image of the communist hero, which did not correspond to the literary and political canon, caused acute criticism and was the reason for suppressing the work. The results of the study give ground to consider the version of the play Time, forward! of 1934, a valuable source reflecting the characters, emotions and feelings of the heroes of its time.

Unsupervised learning of optical flow in a multi-frame dynamic environment using temporal dynamic modeling

For visual estimation of optical flow, which is crucial for various vision analyses, unsupervised learning by view synthesis has emerged as a promising alternative to supervised methods because the ground-truth flow is not readily available in many cases. However, unsupervised learning is likely to be unstable when pixel tracking is lost via occlusion and motion blur, or pixel correspondence is impaired by variations in image content and spatial structure over time. Recognizing that dynamic occlusions and object variations usually exhibit a smooth temporal transition in natural settings, we shifted our focus to model unsupervised learning optical flow from multi-frame sequences of such dynamic scenes. Specifically, we simulated various dynamic scenarios and occlusion phenomena based on Markov property, allowing the model to extract motion laws and thus gain performance in dynamic and occluded areas, which diverges from existing methods without considering temporal dynamics. In addition, we introduced a temporal dynamic model based on a well-designed spatial-temporal dual recurrent block, resulting in a lightweight model structure with fast inference speed. Assuming the temporal smoothness of optical flow, we used the prior motions of adjacent frames to supervise the occluded regions more reliably. Experiments on several optical flow benchmarks demonstrated the effectiveness of our method, as the performance is comparable to several state-of-the-art methods with advantages in memory and computational overhead.

Why Does Anything Exist?

Rasmussen develops a new answer to the question, "Why does anything exist?" He begins by describing a puzzle about how anything can exist. The puzzle motivates the quest to explain things as far as one can. To solve the puzzle, Rasmussen describes a sequence of scenes in a story about existence. The story brings to light a three-pronged explanation of existence: (i) things exist because it is impossible for nothing to have existed, (ii) it is impossible for nothing to have existed because there is a foundational reality that cannot not exist, and (iii) such a foundation would have a certain nature—to be specified--that allows it to be foundational. Rasmussen considers how this theory of fundamental reality can incorporate other large scale theories, including Platonism, axiarchism, and naturalism.

A Pipeline for Story Visualization from Natural Language

Generating automatic visualization from natural language texts is an important task for promoting language learning and literacy development for young children and language learners. However, translating a text into a coherent visualization matching its relevant keywords is a challenging problem. To tackle this issue, we proposed a robust story visualization pipeline ranging from NLP and relation extraction to image sequence generation and alignment. First, we applied a shallow semantic representation of the text where we extracted concepts including relevant characters, scene objects, and events in an appropriate format. We also distinguished between simple and complex actions. This distinction helped to realize an optimal visualization of the scene objects and their relationships according to the target audience. Second, we utilized an image generation framework along with different versions to support the visualization task efficiently. Third, we used CLIP similarity function as a semantic relevance metric to check local and global coherence to the whole story. Finally, we validated the scene sequence to compose a final visualization using the different versions for various target audiences. Our preliminary results showed considerable effectiveness in adopting such a pipeline for a coarse visualization task that can subsequently be enhanced.

Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles

Self-driving cars and trucks, autonomous vehicles ( <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">av</small> s), should not be accepted by regulatory bodies and the public until they have much higher confidence in their safety and reliability — which can most practically and convincingly be achieved by testing. But existing testing methods are inadequate for checking the end-to-end behaviors of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">av</small> controllers against complex, real-world corner cases involving interactions with multiple independent agents such as pedestrians and human-driven vehicles. While test-driving <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">av</small> s on streets and highways fails to capture many rare events, existing simulation-based testing methods mainly focus on simple scenarios and do not scale well for complex driving situations that require sophisticated awareness of the surroundings. To address these limitations, we propose a new fuzz testing technique, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoFuzz</i> , which can leverage widely-used <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">av</small> simulators’ API grammars to generate semantically and temporally valid complex driving scenarios (sequences of scenes). To efficiently search for traffic violations-inducing scenarios in a large search space, we propose a constrained neural network (NN) evolutionary search method to optimize <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoFuzz</i> . Evaluation of our prototype on one state-of-the-art learning-based controller, two rule-based controllers, and one industrial-grade controller in five scenarios shows that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoFuzz</i> efficiently finds hundreds of traffic violationsin high-fidelity simulation environments. For each scenario, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoFuzz</i> can find on average 10-39% more unique traffic violationsthan the best-performing baseline method. Further, fine-tuning the learning-based controller with the traffic violationsfound by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AutoFuzz</i> successfully reduced the traffic violationsfound in the new version of the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">av</small> controller software.

The Sequential Schematic Scene-building Theory in Dan Brown’s The Da Vinci Code: A Cognitive Semantic Study

Everything in the universe is movable or being moved, starting from galaxies and planets to the smallest object, such as the atom. But how can we conceptualize the motion of such entities and what are the basic elements of sequential scene? To do that, a theory of the sequential schematic scene-building has developed to treat this issue. This theory deals with the basic dynamic, on-line, or real-time perceptual processes by which we build a scene. The study focuses on the schematic, not conceptual, elements of sequential scenes which deal with the highly abstracted, primitive system. Such system is considered as the skeleton or building blocks of any sequential scene. To make the theory more applicable, the study selects a scene from Dan Brown’s The Da Vinci Code to be analyzed in terms of the theory developed. The study arrived at a conclusion that the moveable scene can include a group of sequential schematic structures, such as the mover, motion, causality, geometrical structures, and containment. All these elements work together sequentially in the sense that they are inherently consolidated.

Vehicle trajectory data extraction from the horizontal curves of mountainous roads

ABSTRACT Trajectory data is essential for understanding the driver-vehicle-road interaction which is crucial for safety and operational assessment. The video image-processing technique is useful to collect such data on the entire traffic stream. Existing techniques assume that the road is planar which limits the application of the video-based traffic data collection to the plain terrains. With the objective to collect trajectory data from mountainous terrain, the present study proposes an approach to convert the pixel coordinates into real-world coordinates. The hypothesis was that a non-planar scene could be converted into a sequence of piece-wise planar scenes with a separate projection matrix for each planar region. The analysis shows that the proposed method could effectively transform the pixel coordinates into the real-world coordinates. Comparison of the estimated and observed speed/path profiles indicate the adequacy of the proposed method in collecting video-based trajectory data from mountainous terrain.

Learning Scene Dynamics from Point Cloud Sequences

Understanding 3D scenes is a critical prerequisite for autonomous agents. Recently, LiDAR and other sensors have made large amounts of data available in the form of temporal sequences of point cloud frames. In this work, we propose a novel problem—sequential scene flow estimation (SSFE)—that aims to predict 3D scene flow for all pairs of point clouds in a given sequence. This is unlike the previously studied problem of scene flow estimation which focuses on two frames. We introduce the SPCM-Net architecture, which solves this problem by computing multi-scale spatiotemporal correlations between neighboring point clouds and then aggregating the correlation across time with an order-invariant recurrent unit. Our experimental evaluation confirms that recurrent processing of point cloud sequences results in significantly better SSFE compared to using only two frames. Additionally, we demonstrate that this approach can be effectively modified for sequential point cloud forecasting (SPF), a related problem that demands forecasting future point cloud frames. Our experimental results are evaluated using a new benchmark for both SSFE and SPF consisting of synthetic and real datasets. Previously, datasets for scene flow estimation have been limited to two frames. We provide non-trivial extensions to these datasets for multi-frame estimation and prediction. Due to the difficulty of obtaining ground truth motion for real-world datasets, we use self-supervised training and evaluation metrics. We believe that this benchmark will be pivotal to future research in this area. All code for benchmark and models will be made accessible at (https://github.com/BestSonny/SPCM).

The Construalization of Sequential Scenes in Dan Brown’s The Da Vinci Code: A Cognitive Semantic Study

One of the challenges that cognitive linguists face is how the parts of a scene are arranged and how sequential actions are joined together to form a wider scene. The order of actions in a scene is affected by how we see that scene, which is reflected in the language we use. The current study aims to develop a model of construalization to see how sequential scenes in Dan Brown’s The Da Vinci Code are viewed. A group of operations have been suggested under three umbrella terms: saliency, perspective and comparison. The proposed operations can base on either syntactic (such as grammatical saliency) or semantic properties (such as zooming). The study chose 28 actions in two broad scenes. The present study concluded that construalization plays an important role in the viewing arrangement of dynamic scenes. The place of entities in a scene affects our conceptualization of them; whenever the entity moves, our viewpoint changes.

Infrared Small Target Detection via Nonconvex Tensor Fibered Rank Approximation

Infrared small target detection plays an important role in precision guidance, infrared warning, and other applications. The infrared patch-tensor (IPT) model has good detection performance, but some challenges still exist, such as the inaccurate representation of the background rank and poor robustness against noise and sparse interference. In order to solve these problems, a new IPT model is proposed in this article. First, to approximate the tensor rank more reasonably, t-SVD is generalized to multimodal t-SVD, and the tensor fibered rank is introduced. Moreover, the tensor fibered nuclear norm based on the Log operator (LogTFNN) is used to nonconvex approximate tensor fibered rank. Second, to suppress sparse interference such as strong edges and corner points, the prior information is extracted by the local structure tensor. Third, the hypertotal variation (HTV) is used as a joint regularization term to remove noise. Then, the alternating direction method of multipliers (ADMM) is used to solve the model. The proposed algorithm was tested on the 20 single-frame infrared images and six sequences of real scenes. Lots of experiments demonstrate that this algorithm has the robustness to noise and different scenes. Different evaluation metrics also show that the proposed algorithm has a significant superiority in detection performance compared with various state-of-the-art methods.

A Pipeline for Automated Processing of Declassified Corona KH-4 (1962–1972) Stereo Imagery

The Corona KH-4 reconnaissance satellite missions from 1962-1972 acquired panoramic stereo imagery with high spatial resolution of 1.8-7.5 m. The potential of 800,000+ declassified Corona images has not been leveraged due to the complexities arising from handling of panoramic imaging geometry, film distortions and limited availability of the metadata required for georeferencing of the Corona imagery. This paper presents Corona Stereo Pipeline (CoSP): A pipeline for processing of Corona KH-4 stereo panoramic imagery. CoSP utlizes a deep learning based feature matcher SuperGlue to automatically match features point between Corona KH-4 images and recent satellite imagery to generate Ground Control Points (GCPs). To model the imaging geometry and the scanning motion of the panoramic KH-4 cameras, a rigorous camera model consisting of modified collinearity equations with time dependent exterior orientation parameters is employed. The results show that using the entire frame of the Corona image, bundle adjustment using well-distributed GCPs results in an average standard deviation (SD) of less than 2 pixels. The distortion pattern of image residuals of GCPs and y-parallax in epipolar resampled images suggest that film distortions due to long term storage as likely cause of systematic deviations. Compared to the SRTM DEM, the Corona DEM computed using CoSP achieved a Normalized Median Absolute Deviation (NMAD) of elevation differences of ~4 m over an area of approx. 4000 $km^2$. We show that the proposed pipeline can be applied to sequence of complex scenes involving high relief and glacierized terrain and that the resulting DEMs can be used to compute long term glacier elevation changes over large areas.

Continual Learning Approach for Remote Sensing Scene Classification

In this letter, we propose a continual learning approach for a set of sequential scene classification tasks, where each task contains a group of land-cover classes. Our aim is to learn new tasks in a continual way without significantly degrading the performances of the old ones, due to the tricky catastrophic forgetting problem inherent to neural networks. To this end, we propose a neural architecture composed of two trainable modules. The first module learns its weights by discriminating between the land-cover classes within the new task while keeping trace of the old ones. On the other side, the second module tries to maximize the separation between the tasks by learning on task-prototypes stored in a linear memory (one prototype per task). The experimental results on two scene data sets (Merced and Optimal31) confirm the promising capability of the proposed method.

Animate Orientation Based on Visual Landmarks and Scene Recognition

A biologically-inspired approach to robot route following is presented. The ant of the genus Formica rufa (a red forest ant) is used as a model species. These ants actively use collective foraging, unlike many other ant species. The scout ant remembers the route to food and can transmit information about the food location to foraging ants. Foragers can independently reach this place using this data and return home. The basis of the proposed method is the memorization the way by visual landmarks and fuzzy control. The animate path description model consists of a sequence of scenes and includes compass to account for the direction. The behavior of the animate-scout is implemented using an algorithm that simulates the foraging behavior of ants. The animate-forager performs actions to reproduce the route, applying the developed set of rules. The forager behavior is based on the same principles as that of a scout. But the scout remembers the scenes, and the forager recognizes and compares the visible scene and the scene from the route description. The actions of animates are presented in the form of elementary behavioral procedures. Each behavioral procedure is implemented using a finite state machine. The experiments for solving the foraging problem were carried out using a modeling system based on the ROS framework. The simulation results confirm the effectiveness of the proposed method. The method does not require large computing power and advanced sensory capabilities from the robot. It can also be used in reconnaissance and patrol tasks.

Autopsie einer Graphic Novel: Dostoevskijs Prestuplenie i nakazanie (1866) in der Adaption von Alain Korkos und David Zane Mairowitz (2008)

Summary This study offers a comprehensive comparative analysis between F. M. Dostoevskij’s classic Prestuplenie i nakazanie (PSS 6, 1973) and the black-and-white adaptation Crime and Punishment created by Alain Korkos and David Zane Mairowitz (2008). According to author and artist, the adaptation is meant for recipients acquainted with the novel and first readers of the graphic novel alike. With 417 pages of close-printed canonic text turned into a picture-dominated graphic novel of 118 pages, the adaptation – paying attention to the sequence of crucial fictional scenes – follows the storyline from the beginning of the canonic text to the Epilogue. About 95 % of the precisely and densely formulated verbal material contained in captions can easily be traced back to Dostoevskij’s classic. In this respect, the adaptation is „true to the source material“ (Kick 2012). However, as large portions of text concerning Sonja or Sonja and Raskol’nikov are – discernibly purposefully – eliminated, the adaptation turns out to be a revised version of the canonic novel. E.g., Sonja neither reads the Lazarus-chapter to Raskol’nikov nor follows him to Siberia. In contrast to Sonja, Raskol’nikov – with his emotional-mental instability, rebellion against the state of the world, self-authorization to commit crimes – is represented in accordance with the source text. The hermeneutic gain caused through verbal-pictorial reproduction of the classic text – i. e. through meaningful handling of panels and graphic design within the panels – is significantly enlarged through verbal-pictorial material not inspired by the classic text, e. g. a poster showing Edvard Munch’s graphics The Cry. In view of the 21st century’s preference for visual and multimedial presentation, the study encourages the coexistence and ‚co-reception‘ of the literary classic and the graphic novel as well as further forms of cultural processing.

Comment on Strauss et al., 2020: Subsidence analysis of salt tectonics‐driven minibasins (Northern Calcareous Alps, Austria)

Comment on Strauss et al., 2020: Subsidence analysis of salt tectonics‐driven minibasins (Northern Calcareous Alps, Austria)