Scene Changes Research Articles

Optical Encryption Using Attention-Inserted Physics-Driven Single-Pixel Imaging

Optical encryption based on single-pixel imaging (SPI) has made great advances with the introduction of deep learning. However, the use of deep neural networks usually requires a long training time, and the networks need to be retrained once the target scene changes. With this in mind, we propose an SPI encryption scheme based on an attention-inserted physics-driven neural network. Here, an attention module is used to encrypt the single-pixel measurement value sequences of two images, together with a sequence of cryptographic keys, into a one-dimensional ciphertext signal to complete image encryption. Then, the encrypted signal is fed into a physics-driven neural network for high-fidelity decoding (i.e., decryption). This scheme eliminates the need for pre-training the network and gives more freedom to spatial modulation. Both simulation and experimental results have demonstrated the feasibility and eavesdropping resistance of this scheme. Thus, it will lead SPI-based optical encryption closer to intelligent deep encryption.

Wie mobil ist der Erzähler? Beobachtungen zu metaleptischen Szenenwechseln in Antike, Mittelalter und Renaissance

Abstract In a type of metalepsis common from antiquity to modern times, the narrator puts himself in direct contact with the narrated world by visualising the change of scene as a movement between two places or (groups of) characters. This article concentrates on the spatial aspect of this contact and investigates the question to what extent the narrator claims mobility in the narrated space. As the examples discussed show, there are striking differences between the epochs, which are significant for a historicised understanding of storytelling in general. While ancient (i.e., Greek and Latin) narrators sometimes transfer themselves to specific places, medieval vernacular scene shift formulae usually refer to characters, who are envisaged in a performative co-presence with the narrator and the addressees. This situation changes in the Italian Renaissance (Boccaccio, Ariosto), when narrators start to emphasise their mobility again, which they simultaneously enrich with new implications.

Early development of saliency-driven attention through object manipulation

In the first years of life, infants progressively develop attention selection skills to gather information from visually clustered environments. As young as newborns, infants are sensitive to the distinguished differences in color, orientation, and luminance, which are the components of visual saliency. However, we know little about how saliency-driven attention emerges and develops socially through everyday free-viewing experiences. The present work assessed the saliency change in infants' egocentric scenes and investigated the impacts of manual engagements on infant object looking in the interactive context of object play. Thirty parent-infant dyads, including infants in two age groups (younger: 3- to 6-month-old; older: 9- to 12-month-old), completed a brief session of object play. Infants' looking behaviors were recorded by the head-mounted eye-tracking gear, and both parents' and infants' manual actions on objects were annotated separately for analyses. The present findings revealed distinct attention mechanisms that underlie the hand-eye coordination between parents and infants and within infants during object play: younger infants are predominantly biased toward the characteristics of the visual saliency accompanying the parent's handled actions on the objects; on the other hand, older infants gradually employed more attention to the object, regardless of the saliency in view, as they gained more self-generated manual actions. Taken together, the present work highlights the tight coordination between visual experiences and sensorimotor competence and proposes a novel dyadic pathway to sustained attention that social sensitivity to parents' hands emerges through saliency-driven attention, preparing infants to focus, follow, and steadily track moving targets in free-flow viewing activities.

A high-accuracy multi-temporal laser point cloud registration method for providing a detailed expression of the urban environment

In consideration of the significant changes in urban environmental scenes and the impact of factors such as signal obstruction on positioning, the spatial positions of multi-temporal Mobile Laser Scanning (MLS) point clouds collected in the same area are inconsistent, which makes it challenging to provide a complete expression of the urban road environment. To address this issue, a precise registration algorithm based on pole-like targets is proposed. The core steps include registration primitive extraction: a cylindrical spatial neighborhood is established based on the distinctive key points, and precise extraction of pole-like geospatial point cloud data is performed using the Euclidean clustering method with the normal curvature constraint. Achieving optimal spatial transformation: a circle center fitting method with additional Mean Absolute Error (MAE) loss function constraints was developed based on the stable elemental unit which was utilized to accurately construct homonymous feature point pairs. Validation results from multiple real urban point cloud datasets collected from different experimental areas and platforms demonstrate that our method achieves superior registration performance, with mean residuals below 5 cm in both the planar and elevation directions. The proposed method exhibits strong universality and registration robustness for multi-temporal point cloud data of the homologous or cross-sources.

AIPT: Adaptive information perception for online multi-object tracking

Information perception is crucial in MOT tasks. Recent approaches use positional, motion, and appearance information to model object states. However, in scenes involving camera motion, tracking tasks suffer from image distortion, trajectory loss, and mismatching issues. In this paper, we propose Adaptive Information Perception for Online Multi-Object Tracking, abbreviated as AIPT. AIPT consists of an Adaptive Motion Perception Module (AMPM) and an Asymmetric Information Suppression Module (AISM). In AMPM, we design an Adaptive Image Distortion Recovery Module (AIDRM) to perceive distortions in unknown scenes, allowing the tracker to autonomously recover distorted images as the scene changes. By designing the Information-Guided Trajectory Restoration Module (IGTRM), the tracker learns object motion states from prior information and constructs accurate reconstruction information during trajectory loss. Furthermore, our AISM module utilizes masking information to suppress potential relationships between asymmetric objects, thereby enhancing the ability of tracker to handle mismatches. Both AMPM and AISM exhibit excellent scalability, seamlessly integrating with most advanced tracking methods. Ultimately, our AIPT achieves leading performance on multiple benchmark platforms, including MOT17, MOT20, and KITTI.

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity.

Roadside camera-driven 3D object detection is a crucial task in intelligent transportation systems, which extends the perception range beyond the limitations of vision-centric vehicles and enhances road safety. While previous studies have limitations in using only depth or height information, we find both depth and height matter and they are in fact complementary. The depth feature encompasses precise geometric cues, whereas the height feature is primarily focused on distinguishing between various categories of height intervals, essentially providing semantic context. This insight motivates the development of Complementary-BEV (CoBEV), a novel end-to-end monocular 3D object detection framework that integrates depth and height to construct robust BEV representations. In essence, CoBEV estimates each pixel's depth and height distribution and lifts the camera features into 3D space for lateral fusion using the newly proposed two-stage complementary feature selection (CFS) module. A BEV feature distillation framework is also seamlessly integrated to further enhance the detection accuracy from the prior knowledge of the fusion-modal CoBEV teacher. We conduct extensive experiments on the public 3D detection benchmarks of roadside camera-based DAIR-V2X-I and Rope3D, as well as the private Supremind-Road dataset, demonstrating that CoBEV not only achieves the accuracy of the new state-of-the-art, but also significantly advances the robustness of previous methods in challenging long-distance scenarios and noisy camera disturbance, and enhances generalization by a large margin in heterologous settings with drastic changes in scene and camera parameters. For the first time, the vehicle AP score of a camera model reaches 80% on DAIR-V2X-I in terms of easy mode. The source code will be made publicly available at CoBEV.

Smart Cook : An AI and Machine Learning Powered Culinary

By utilizing the power of artificial intelligence (AI) and machine learning (ML), the SmartCook system transforms how we approach cooking, marking a significant advancement in the field of culinary arts. SmartCook provides a cutting-edge solution for both inexperienced and seasoned chefs as the culinary scene changes. An all-inclusive culinary helper, SmartCook draws from a huge database of recipes, ingredients, and cooking methods. With the help of its AI skills, it can comprehend user preferences, dietary constraints, and the components that are accessible to create personalized recipe recommendations. Additionally, it provides in-the-moment instruction during cooking, supporting users in producing food with the best flavour and texture. Smart Cook’s machine learning component continuously enhances its predictions and suggestions by gathering information from user interactions and comments. It changes as dietary trends do.

Contrastive Scene Change Representation Learning for High-Resolution Remote Sensing Scene Change Detection

Contrastive Scene Change Representation Learning for High-Resolution Remote Sensing Scene Change Detection

Neuromorphic Imaging With Joint Image Deblurring and Event Denoising.

Neuromorphic imaging reacts to per-pixel brightness changes of a dynamic scene with high temporal precision and responds with asynchronous streaming events as a result. It also often supports a simultaneous output of an intensity image. Nevertheless, the raw events typically involve a large amount of noise due to the high sensitivity of the sensor, while capturing fast-moving objects at low frame rates results in blurry images. These deficiencies significantly degrade human observation and machine processing. Fortunately, the two information sources are inherently complementary - events with microsecond-level temporal resolution, which are triggered by the edges of objects recorded in a latent sharp image, can supply rich motion details missing from the blurry one. In this work, we bring the two types of data together and introduce a simple yet effective unifying algorithm to jointly reconstruct blur-free images and noise-robust events in an iterative coarse-to-fine fashion. Specifically, an event-regularized prior offers precise high-frequency structures and dynamic features for blind deblurring, while image gradients serve as a kind of faithful supervision in regulating neuromorphic noise removal. Comprehensively evaluated on real and synthetic samples, such a synergy delivers superior reconstruction quality for both images with severe motion blur and raw event streams with a storm of noise, and also exhibits greater robustness to challenging realistic scenarios such as varying levels of illumination, contrast and motion magnitude. Meanwhile, it can be driven by much fewer events and holds a competitive edge at computational time overhead, rendering itself preferable as available computing resources are limited. Our solution gives impetus to the improvement of both sensing data and paves the way for highly accurate neuromorphic reasoning and analysis.

Piloting a simplified bio-behavioural survey methodology, the BBS-Lite, among people who inject drugs in Georgia

BackgroundDuring 2021 and 2023 two simplified Biological and Behavioural Study (BBS-Lite) surveys, and in 2022 one Standard Integrated Biological and Behavioural Study (IBBS), were conducted among people who inject drugs in seven cities in Georgia. From these, an opportunity to compare the implementation of these survey methods and results was able to be gained. MethodsThe two survey types were compared to find points of similarity and difference in their methodologies. The methodologies of the IBBS and BBS-Lite studies shared many characteristics, including the cities where they were implemented, recruitment criteria, sample sizes, and common questionnaire items. All studies were multi-centre cross-sectional involving administration of a face-to-face behavioural questionnaire and collection of biological specimens for testing of HIV, syphilis, hepatitis B virus (HBV), and hepatitis C virus (HCV). The main differences were in the sampling methods. The IBBS utilised respondent-driven sampling (RDS) while participants of the BBS-Lite studies were enrolled through consecutive recruitment at the harm reduction (HR) programme sites and on outreach and through snowball sampling. We compared the results from each study as well as the implementation modalities modalities such as time taken and budgetary requirements, and the complexity of implementation. ResultsConsiderably less time was required for recruitment, as well as for interviewing, data entry (4 times less) and the analysis for the BSS-Lite studies compared to the IBBS. The BSS-Lite study budgets were at least 2.5 times less than of the IBBS study.The recruited samples were comparable for age distribution, median age at first injection, the last drug injected, sharing of drug injecting equipment or receiving opioid agonist maintenance treatment (OAMT) during the last 12 months. HIV and HCV prevalence were similar including for stratifications by age, client status and city of recruitment. ConclusionOur findings have demonstrated that if implemented on a regular basis, the BBS-Lite can be a complementary solution for systematic data collection, filling surveillance gaps and addressing the challenges that persist in obtaining important data on people who inject drugs (PWID) between IBBS rounds in the country. The methodology is recommended for testing in other settings and in other key populations. In addition, the data collected on a routine base can help the harm reduction (HR) program to better understand the changes in the drug scene and observe new trends in HIV risks and drug injecting behaviours, possible barriers for access to harm reduction, drug treatment, and HIV and/or viral hepatitis testing and treatment services.

Stray bird species are reference points of the initial stages of changing areas in conditions of directional climate change (on The example Eastern Siberia)

We have shown that Eastern Siberia has now become the scene of climate change, which is determined by its location – the center of Inner Asia. It was found that as a result of increased solar activity on the western edge of Eurasia, the latitudinal transfer of air masses was replaced by a meridional (North Atlantic sector) one. Later, such a shift was observed in the eastern (Pacific) sector of Asia. As a result, a hotbed of elevated surface air temperature was formed in Eastern Siberia and adjacent areas of Mongolia and China, and distant bird evictions to the north, east and west were observed. It is shown that such flights are the first stage of the transformation of bird areas. In the first half of the 20th century, weak western and eastern bird streams have already reached of the hollow Lake Baikal. By the middle of the 20th century, 60 new bird species had been recorded here, but their numbers were insignificant. Subsequently, there were clear trends among them towards further expansion of their areas. It is proved that long-range flights of new species are the first benchmarks indicating the upcoming changes in bird areas. Their consequences should be taken into account in economic activities.

Analyzing the Impact of Scene Transitions on Indoor Camera Localization through Scene Change Detection in Real-Time

Analyzing the Impact of Scene Transitions on Indoor Camera Localization through Scene Change Detection in Real-Time

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification

This research proposes a deep learning method for classifying student behavior in classrooms that follow the professional learning community teaching approach. We collected data on five student activities: hand-raising, interacting, sitting, turning around, and writing. We used the sum of absolute differences (SAD) in the LUV color space to detect scene changes. The K-means algorithm was then applied to select keyframes using the computed SAD. Next, we extracted features using multiple pretrained deep learning models from the convolutional neural network family. The pretrained models considered were InceptionV3, ResNet50V2, VGG16, and EfficientNetB7. We leveraged feature fusion, incorporating optical flow features and data augmentation techniques, to increase the necessary spatial features of selected keyframes. Finally, we classified the students’ behavior using a deep sequence model based on the bidirectional long short-term memory network with an attention mechanism (BiLSTM-AT). The proposed method with the BiLSTM-AT model can recognize behaviors from our dataset with high accuracy, precision, recall, and F1-scores of 0.97, 0.97, and 0.97, respectively. The overall accuracy was 96.67%. This high efficiency demonstrates the potential of the proposed method for classifying student behavior in classrooms.

Thoughts on the Joanine Style in 18th-Century Luso-Brazilian Religious Decorations

The article aims to stimulate reflections on the gilded and polychrome woodwork of the Joanine Style found in 18th-century Luso-Brazilian churches and chapels. It begins with an overview of the cultural and artistic context fostered by the patronage of Dom João V (1689-1750), whose actions brought about changes in the Portuguese art scene, influencing Brazil. Through the analysis of these aspects, the article examines the influence of Portuguese art on Brazilian religious decoration between approximately 1720-1760. It highlights the presumed means used to enrich the formal repertoire of executed works, the formal characterization of remaining pieces, and the involvement of various artists involved in this process. Within this scope, the article also prompts reflections on the wooden ornamentation of the Joanine style in Brazil, a theme that is still insufficiently researched.

Landmark Topology Descriptor-Based Place Recognition and Localization under Large View-Point Changes.

Accurate localization between cameras is a prerequisite for a vision-based heterogeneous robot systems task. The core issue is how to accurately perform place recognition from different view-points. Traditional appearance-based methods have a high probability of failure in place recognition and localization under large view-point changes. In recent years, semantic graph matching-based place recognition methods have been proposed to solve the above problem. However, these methods rely on high-precision semantic segmentation results and have a high time complexity in node extraction or graph matching. In addition, methods only utilize the semantic labels of the landmarks themselves to construct graphs and descriptors, making such approaches fail in some challenging scenarios (e.g., scene repetition). In this paper, we propose a graph-matching method based on a novel landmark topology descriptor, which is robust to view-point changes. According to the experiment on real-world data, our algorithm can run in real-time and is approximately four times and three times faster than state-of-the-art algorithms in the graph extraction and matching phases, respectively. In terms of place recognition performance, our algorithm achieves the best place recognition precision at a recall of 0-70% compared with classic appearance-based algorithms and an advanced graph-based algorithm in the scene of significant view-point changes. In terms of positioning accuracy, compared to the traditional appearance-based DBoW2 and NetVLAD algorithms, our method outperforms by 95%, on average, in terms of the mean translation error and 95% in terms of the mean RMSE. Compared to the state-of-the-art SHM algorithm, our method outperforms by 30%, on average, in terms of the mean translation error and 29% in terms of the mean RMSE. In addition, our method outperforms the current state-of-the-art algorithm, even in challenging scenarios where the benchmark algorithms fail.

Misidentifying illuminant changes in natural scenes due to failures in relational colour constancy.

The colours of surfaces in a scene may not appear constant with a change in the colour of the illumination. Yet even when colour constancy fails, human observers can usually discriminate changes in lighting from changes in surface reflecting properties. This operational ability has been attributed to the constancy of perceived colour relations between surfaces under illuminant changes, in turn based on approximately invariant spatial ratios of cone photoreceptor excitations. Natural deviations in these ratios may, however, lead to illuminant changes being misidentified. The aim of this work was to test whether such misidentifications occur with natural scenes and whether they are due to failures in relational colour constancy. Pairs of scene images from hyperspectral data were presented side-by-side on a computer-controlled display. On one side, the scene underwent illuminant changes and on the other side, it underwent the same changes but with images corrected for any residual deviations in spatial ratios. Observers systematically misidentified the corrected images as being due to illuminant changes. The frequency of errors increased with the size of the deviations, which were closely correlated with the estimated failures in relational colour constancy.

Temperate grassland soil nitrifiers are more sensitive to nitrogen addition than simulated warming

Soil nitrogen transformations are influenced by both global climate change and human activities. Nitrification, affected by ammonia-oxidizing and nitrite-oxidizing microorganisms, is one of the crucial N transformation processes. However, the impact of environmental change on the diversity and community structure of these microorganisms in grassland ecosystems remains poorly understood. This study explored the alpha diversity and community structures of N-cycling microorganisms in response to simulated warming and N addition in a temperate steppe. The results showed that N addition significantly increased the taxon richness of ammonia-oxidizing bacteria and nitrite-oxidizing bacteria but significantly decreased the diversity of total archaea. Also, N addition significantly changed the community structures of ammonia-oxidizing bacteria, nitrite-oxidizing bacteria, and total archaea and bacteria. However, warming had no effect on the whole microbiome alpha and beta diversity. The alternations of microbial communities were mainly induced by changes in soil pH and inorganic N concentrations. This study illustrates that the temperate grassland microbial communities are more sensitive to N addition than simulated warming under future climate change scenes.

Deep Learning Approach for Human Action Recognition Using a Time Saliency Map Based on Motion Features Considering Camera Movement and Shot in Video Image Sequences

In this article, a hierarchical method for action recognition based on temporal and spatial features is proposed. In current HAR methods, camera movement, sensor movement, sudden scene changes, and scene movement can increase motion feature errors and decrease accuracy. Another important aspect to take into account in a HAR method is the required computational cost. The proposed method provides a preprocessing step to address these challenges. As a preprocessing step, the method uses optical flow to detect camera movements and shots in input video image sequences. In the temporal processing block, the optical flow technique is combined with the absolute value of frame differences to obtain a time saliency map. The detection of shots, cancellation of camera movement, and the building of a time saliency map minimise movement detection errors. The time saliency map is then passed to the spatial processing block to segment the moving persons and/or objects in the scene. Because the search region for spatial processing is limited based on the temporal processing results, the computations in the spatial domain are drastically reduced. In the spatial processing block, the scene foreground is extracted in three steps: silhouette extraction, active contour segmentation, and colour segmentation. Key points are selected at the borders of the segmented foreground. The last used features are the intensity and angle of the optical flow of detected key points. Using key point features for action detection reduces the computational cost of the classification step and the required training time. Finally, the features are submitted to a Recurrent Neural Network (RNN) to recognise the involved action. The proposed method was tested using four well-known action datasets: KTH, Weizmann, HMDB51, and UCF101 datasets and its efficiency was evaluated. Since the proposed approach segments salient objects based on motion, edges, and colour features, it can be added as a preprocessing step to most current HAR systems to improve performance.

Mapping property redevelopment via GeoAI: Integrating computer vision and socioenvironmental patterns and processes

Domain knowledge of social and environmental sciences is generally derived from less structured small data and/or small models. The integration of deep learning with socioenvironmental and geographical patterns and processes is still in an early phase. This study proposes a flexible framework that synthesizes computer vision with patterns and processes of geographical phenomena (e.g., urban redevelopment) via deep convolutional neural networks and spatiotemporal neighborhoods, respectively. Meanwhile, undesirable visual cues (e.g., temporary objects such as cars) cause false alarms in urban change detection. Thus, a masked deep pyramid similarity model (i.e., the computer vision model) is proposed to minimize the negative impact of nonbuilding changes. This pipeline has robustness against obfuscation of undesirable street scene changes and elasticity of geographical knowledge representations in mapping property redevelopment. This model is reproducible and adaptable due to the wide availability of SVI and flexibility of the framework. The results suggest that the domain-driven rules and nonbuilding change masking mechanism can significantly increase the accuracy of a computer vision model. We also find that urban redevelopment should be understood as a locally tuned back-to-the-city process with weak replicability. Hybrid gentrification (i.e., the combination of “seesaw” gentrification and continuous gentrification) is observed globally with local variations. Takeaway for practiceFirst, the population of those affected should be carefully identified and the benefits of gentrification need to be redistributed on a larger scale. For Auckland, decision-makers should not divert attention away from low-income tenants in outer suburbs that slipped below the radar of local government and scholars. Second, more attention should be given to less resourcing neighborhoods in constantly gentrifying neighborhoods when governments formulate and implement housing subsidy policies. Locally, incessant redevelopment in Auckland CBD, New Market and Remuera, etc., should be contained.