Correspondence Problem Research Articles

Dynamic Scene Representation for Docking in Urban Waters Using a Stereo Camera

Abstract To navigate and dock safely in urban waters, an autonomous ferry must assess its safe navigation paths by identifying navigable free space and potential obstacles. A deep understanding of its surroundings is crucial for situational awareness. Knowing which objects are static and dynamic is an indispensable ingredient for path planning algorithms and collision avoidance systems, and is especially useful for docking operations, as moving obstacles can interfere with the optimal path to the dock. To handle all types of obstacles, we present a novel dynamic scene representation for docking in urban waters using a short baseline stereo camera. Obstacles protruding from the water surface are represented with vertically oriented rectangles, known as stixels. The stixels are tracked over consecutive frames with dense optical flow. Together with stereo correspondences, their 3D motion is estimated. Stixels representing the same object are grouped together into line segments which are classified as either dynamic or static based on the motion of its associated stixels. Our dynamic representation presents objects in the 3D camera coordinate system as a bird’s eye view map. Additionally, stereo depth uncertainty can significantly impact depth estimates and is considered using a Kalman filter, providing smooth position and velocity estimates. We test our dynamic scene representation on a real docking scenario recorded with an autonomous ferry using a stereo camera. We evaluate the accuracy of the reconstructed 3D scene points and the velocity estimates of the dynamic objects using LiDAR and GNSS.

Performance Comparison of Stereo Correspondence Algorithms in Dense Image Matching

Performance Comparison of Stereo Correspondence Algorithms in Dense Image Matching

Long-term reprojection loss for self-supervised monocular depth estimation in endoscopic surgery

Aim: Depth information plays a key role in enhanced perception and interaction in image-guided surgery. However, it is difficult to obtain depth information with monocular endoscopic surgery due to a lack of reliable cues for perceiving depth. Although there are reprojection loss-based self-supervised learning techniques to estimate depth and pose, the temporal information from the adjacent frames is not efficiently utilized to handle occlusion in surgery. Methods: We design long-term reprojection loss (LT-RL) self-supervised monocular depth estimation techniques by integrating longer temporal sequences into reprojection to learn better perception and to address occlusion artifacts in image-guided laparoscopic and robotic surgery. For this purpose, we exploit four temporally adjacent source frames before and after the target frame, where conventional reprojection loss uses two adjacent frames. The pixels that are visible in the target frame but occluded in the immediate two adjacent frames will produce the inaccurate depth but a higher chance to appear in the four adjacent frames during the calculation of minimum reprojection loss. Results: We validate LT-RL on the benchmark surgical datasets of Stereo correspondence and reconstruction of endoscopic data (SCARED) and Hamlyn to compare the performance with other state-of-the-art depth estimation methods. The experimental results show that our proposed technique yields 2%-4% better root-mean-squared error (RMSE) over the baselines of vanilla reprojection loss. Conclusion: Our LT-RL self-supervised depth and pose estimation technique is a simple yet effective method to tackle occlusion artifacts in monocular surgical video. It does not add any training parameters, making it flexible for integration with any network architecture and improving the performance significantly.

Self-Supervised Monocular Depth Estimation for Endoscopic Imaging.

Endoscopy holds a pivotal role in the early detection and treatment of diverse diseases, with artificial intelligence (AI)-assisted methods increasingly gaining prominence in disease screening. Among them, the depth estimation from endoscopic sequences is crucial for a spectrum of AI-assisted surgical techniques. However, the development of endoscopic depth estimation algorithms presents a formidable challenge due to the unique environmental intricacies and constraints within the dataset. This paper proposes a self-supervised depth estimation network to comprehensively explore the brightness changes in endoscopic images, and fuse different features at multiple levels to achieve an accurate prediction of endoscopic depth. First, a FlowNet is designed to evaluate the brightness changes of adjacent frames by calculating the multi-scale structural similarity. Second, a feature fusion module is presented to capture multi-scale contextual information. Experiments show that the average accuracy of the algorithm is 97.03% in the Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED dataset). Based on the training parameters of the SCARED dataset, the algorithm achieves superior performance on the other two datasets (EndoSLAM and KVASIR dataset), indicating that the algorithm has good generalization performance.

SGDFormer: One-stage transformer-based architecture for cross-spectral stereo image guided denoising

Cross-spectral image guided denoising has shown its great potential in recovering clean images with rich details, such as using the near-infrared image to guide the denoising process of the visible one. To obtain such image pairs, a feasible and economical way is to employ a stereo system, which is widely used on mobile devices. In this case, it is necessary to first model the stereo correspondence between two images before guided denoising. Current works attempt to generate an aligned guidance image to handle the disparity. However, due to occlusion, spectral differences and noise degradation, the aligned guidance image generally exists ghosting and artifacts, leading to an unsatisfactory denoised result. To address this issue, we propose a one-stage transformer-based architecture, named SGDFormer, for cross-spectral Stereo image Guided Denoising. The architecture integrates the correspondence modeling and feature fusion of stereo images into a unified network. Our transformer block contains a noise-robust cross-attention (NRCA) module and a spatially variant feature fusion (SVFF) module. The NRCA module captures the long-range correspondence of two images in a coarse-to-fine manner to alleviate the interference of noise. The SVFF module further enhances salient structures and suppresses harmful artifacts through dynamically selecting useful information. Thanks to the above design, our SGDFormer can restore artifact-free images with fine structures, and achieves state-of-the-art performance on various datasets. Additionally, our SGDFormer can be extended to handle other unaligned cross-model guided restoration tasks such as guided depth super-resolution.

Text-guided distillation learning to diversify video embeddings for text-video retrieval

Conventional text-video retrieval methods typically match a video with a text on a one-to-one manner. However, a single video can contain diverse semantics, and text descriptions can vary significantly. Therefore, such methods fail to match a video with multiple texts simultaneously. In this paper, we propose a novel approach to tackle this one-to-many correspondence problem in text-video retrieval. We devise diverse temporal aggregation and a multi-key memory to address temporal and semantic diversity, consequently constructing multiple video embedding paths from a single video. Additionally, we introduce text-guided distillation learning that enables each video path to acquire meaningful distinct competencies in representing varied semantics. Our video embedding approach is text-agnostic, allowing the prepared video embeddings to be used continuously for any new text query. Experiments show our method outperforms existing methods on four datasets. We further validate the effectiveness of our designs with ablation studies and analyses on diverse video embeddings.

Behavior of Correlation Functions in the Dynamics of the Multiparticle Quantum Arnol'd Cat.

The multi-particle Arnol'd cat is a generalization of the Hamiltonian system, both classical and quantum, whose period evolution operator is the renowned map that bears its name. It is obtained following the Joos-Zeh prescription for decoherence by adding a number of scattering particles in the configuration space of the cat. Quantization follows swiftly if the Hamiltonian approach, rather than the semiclassical approach, is adopted. The author has studied this system in a series of previous works, focusing on the problem of quantum-classical correspondence. In this paper, the dynamics of this system are tested by two related yet different indicators: the time autocorrelation function of the canonical position and the out-of-time correlator of position and momentum.

Assessing brain neuroplasticity: Surface morphometric analysis of cortical changes induced by Quadrato motor training.

Morphological markers for brain plasticity are still lacking and their findings are challenged by the extreme variability of cortical brain surface. Trying to overcome the "correspondence problem," we applied a landmark-free method (the generalized procrustes surface analysis (GPSA)) for investigating the shape variation of cortical surface in a group of 40 healthy volunteers (i.e., the practice group) subjected to daily motor training known as Quadrato motor training (QMT). QMT is a sensorimotor walking meditation that aims at balancing body, cognition, and emotion. More specifically, QMT requires coordination and attention and consists of moving in one of three possible directions on corners of a 50 × 50 cm2. Brain magnetic resonance images (MRIs) of practice group (acquired at baseline, as well as after 6 and 12 weeks of QMT), were 3D reconstructed and here compared with brain MRIs of six more volunteers never practicing the QMT (naïve group). Cortical regions mostly affected by morphological variations were visualized on a 3D average color-scaled brain surface indicating from higher (red) to lower (blue) levels of variation. Cortical regions interested in most of the shape variations were as follows: (1) the supplementary motor cortex; (2) the inferior frontal gyrus (pars opercolaris) and the anterior insula; (3) the visual cortex; (4) the inferior parietal lobule (supramarginal gyrus and angular gyrus). Our results show that surface morphometric analysis (i.e., GPSA) can be applied to assess brain neuroplasticity processes, such as those stimulated by QMT.

A Stochastic-Geometrical Framework for Object Pose Estimation Based on Mixture Models Avoiding the Correspondence Problem

Pose estimation of rigid objects is a practical challenge in optical metrology and computer vision. This paper presents a novel stochastic-geometrical modeling framework for object pose estimation based on observing multiple feature points. This framework utilizes mixture models for feature point densities in object space and for interpreting real measurements. Advantages are the avoidance to resolve individual feature correspondences and to incorporate correct stochastic dependencies in multi-view applications. First, the general modeling framework is presented, second, a general algorithm for pose estimation is derived, and third, two example models (camera and lateration setup) are presented. Numerical experiments show the effectiveness of this modeling and general algorithm by presenting four simulation scenarios for three observation systems, including the dependence on measurement resolution, object deformations and measurement noise. Probabilistic modeling utilizing mixture models shows the potential for accurate and robust pose estimations while avoiding the correspondence problem.

Problems of correspondence between contractual and tort liability in the Civil law of Ukraine

The article explores the regulation of civil legal relations between parties arising from the breach of contract terms and the infliction of harm in Ukraine, specifically analyzing the correspondence between the grounds for the emergence of such legal relations. Although the topic of the relationship between contractual and tort liability has been addressed by scholars repeatedly, there is still no consensus on resolving this issue in Ukrainian legal doctrine. The doctrine contains many contradictions and gaps, negatively impacting the level of legal science, which is reflected in the quality of national civil legislation. The research analyzes the theoretical foundations of the legal phenomenon of legal liability in the context of defining the legal nature of contractual and tort liability. It examines scientific approaches and general principles underlying Ukrainian civil legislation, focusing on the correspondence between types of liability and the grounds for applying corresponding liability measures. The analysis also delves into Ukrainian legal doctrine regarding contractual and tort liability, highlighting the problems and contradictions that have led to legal uncertainty on the research topic. The current state of Ukrainian legislation indicates the absence of a systematic approach to defining and regulating contractual or tort liability. The Civil Code of Ukraine does not contain such definitions or use such concepts in its provisions. While these concepts are common in legal theory, the array of different, often conflicting, scholarly positions prevents the formation of a unified conclusion. Due to the legal uncertainty arising from theoretical approaches in relating contractual and tort liability, there is still a need for further research on this matter. As a result of this article, an analysis of the imperfections of Ukrainian legal doctrine is conducted, and ways to address legal uncertainty are proposed.

Editing mesh sequences with varying connectivity

Time-varying connectivity of triangle mesh sequences leads to substantial difficulties in their processing. Unlike editing sequences with constant connectivity, editing sequences with varying connectivity requires addressing the problem of temporal correspondence between the frames of the sequence. We present a method for time-consistent editing of triangle mesh sequences with varying connectivity using sparse temporal correspondence, which can be obtained using existing methods. Our method includes a deformation model based on the usage of the sparse temporal correspondence, which is suitable for the temporal propagation of user-specified deformations of the edited surface with respect to the shape and true topology of the surface while preserving the individual connectivity of each frame. Since there is no other method capable of comparable types of editing on time-varying meshes, we compare our method and the proposed deformation model with a baseline approach and demonstrate the benefits of our framework.

CVGSR: Stereo image Super-Resolution with Cross-View guidance

The ability to capture the complementary information of stereo images is critical to the development of stereo image super-resolution. Most existing studies have attempted to integrate reliable stereo correspondence along the polar directions through parallax attention and various fusion strategies. However, most of these approaches ignore the large parallax differences in stereo images, resulting in poor performance of convolutional-based parallax attention in capturing the long-range dependencies between images. In this paper, we propose a novel cross-view guided stereo image super-resolution network (CVGSR) for reconstructing high-resolution stereo image pairs with rich texture details by fully exploiting the complementary nature of stereo image pairs. Specifically, we first deploy a cross-view interaction module (CVIM) to explore intra/cross-view dependencies from local to global to compensate for the incomplete compatibility of information between the left and right views. This module uses a progressive cross-guiding strategy to better merge features from occluded and non-occluded regions. Based on this, an efficient attention Transformer (EAT) is improved to activate more input information and further mine the cross-view complementarity. Furthermore, we design a texture loss to optimize the visual perceptual quality of reconstructed images with sharp boundaries and rich texture details. Extensive experiments on four stereo image datasets demonstrate that the proposed CVGSR achieves a competitive and excellent performance.

Warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography

High speed photography technique is potentially the most effective way to measure the motion parameter of warhead fragment benefiting from its advantages of high accuracy, high resolution and high efficiency. However, it faces challenge in dense objects tracking and 3D trajectories reconstruction due to the characteristics of small size and dense distribution of fragment swarm. To address these challenges, this work presents a warhead fragments motion trajectories tracking and spatio-temporal distribution reconstruction method based on high-speed stereo photography. Firstly, background difference algorithm is utilized to extract the center and area of each fragment in the image sequence. Subsequently, a multi-object tracking (MOT) algorithm using Kalman filtering and Hungarian optimal assignment is developed to realize real-time and robust trajectories tracking of fragment swarm. To reconstruct 3D motion trajectories, a global stereo trajectories matching strategy is presented, which takes advantages of epipolar constraint and continuity constraint to correctly retrieve stereo correspondence followed by 3D trajectories refinement using polynomial fitting. Finally, the simulation and experimental results demonstrate that the proposed method can accurately track the motion trajectories and reconstruct the spatio-temporal distribution of 1.0×103 fragments in a field of view (FOV) of 3.2 m×2.5 m, and the accuracy of the velocity estimation can achieve 98.6%.

Object Detection Using Image Processing For Blind Persons: A Review

Abstract: The fundamental idea of detection of objects for videos is confirming an object's existence in a series of images and maybe pinpointing it specifically for identification. Monitoring an item's existence, location, size, form, and other physical and Changes in time throughout a video sequence is known as object tracking. The challenge of reproducing the designated area in consecutive frames of a set of photographs captured at closely spaced time intervals is referred to as the temporal correspondence problem. and it must be solved in order to do this. The two processes in question are interrelated. The foundation of tracking is detection, which often begins with the detection of objects. To aid and validate tracking, it is frequently required to identify an item repeatedly in a following image sequence. In light of this, the present review describes how object recognition using image processing might assist the blind.

Доклад генерал-майора Д. П. Струкова о постановке военной цензуры в Петрограде и Петроградском военном округе. 1915 г.

The article publishes the report of the chairman of the Petrograd Military Censorship Commission, General D. P. Strukov, to the Chief of the Petrograd Military District on the organization of military censorship in the capital (July 9, 1915). By this time military censorship in Petrograd was a complex and multilevel structure, covering not only the press and correspondence, but also theater, cinema, public speeches and reports. Under the "Provisional Regulations on Military Censorship," adopted with the outbreak of World War I, censors had broad powers and could ban virtually any printed material, stop the forwarding of any correspondence, and prohibit any speech or theatrical production. However, this did not avoid numerous mistakes and contradictory decisions when censoring newspapers and magazines. It was not uncommon for one censor to authorize an article for publication and another to prohibit it. Significant difficulties also arose in the censorship of postal and telegraphic correspondence. Due to the widespread and chronic shortage of qualified personnel, postal and telegraph offices and post offices did not have time to review thousands of letters and telegrams, both internal and external. The "backlogs" of correspondence that had already formed in the first month of the war attracted the attention of the command and the censorship leadership. Meetings were held, censors were transferred to the "problem" areas, but the problem was solved slowly, and the "backlogs" generally increased. Letters and telegrams were delayed for months and often did not reach their addressees, which caused great irritation in the society and in the army. Representatives of the censorship called for a multiple increase in the funding of censorship offices and increase in the staff to solve the problem of "deposits" and complete control of internal and external correspondence. The supporter of such a campaign was General Strukov, who proposed in his report of July 9, 1915, to increase the number of censors by 450 people, to raise their salaries, and to establish a full review of all letters and telegrams. In addition, Strukov proposed to change the structure of military censorship in the capital: to subordinate all military censorship commissions of the Petrograd Military District to a single head, subordinate in his turn to the commander of the VI Army. With the help of such "unity of command" the General intended to eliminate the problem of contradictory decisions, unnecessary correspondence and conflicts between departments. Strukov's proposals were realized only partially: in 1915-1916 the number of censors was practically doubled, the maintenance of military censorship venues was improved. However, the problems of censorship - inconsistency, unjustified rigidity, selectivity and constant delay of correspondence - remained for the whole period of the First World War.

Noise-robust Vision-language Pre-training with Positive-negative Learning.

Vision-Language Pre-training (VLP) has shown promising performance in various tasks by learning a generic image-text representation space. However, most existing VLP methods encounter the Noisy Correspondence (NC) problem which refers to wrongly matched image-text pairs harvested from the wild. In this paper, we empirically study the influence of NC on the VLP model and obtain the following two observations. First, the NC will largely degrade the performance in downstream tasks even via fine-tuning, indicating the necessity of handling NC in the pre-training period. Second, the influence of NC varies in different pre-training objectives, suggesting the objective-customized solution for achieving NC robustness. Based on the above observations, we propose a novel NoisE-robust Vision-languagE pRe-training method (NEVER) to endow the VLP model with robustness against NC. In brief, NEVER first divides the training data into clean and noisy subsets in a progressive and adaptive manner. Then NEVER employs the positive learning (PL) and negative learning (NL) on the splits to enjoy model convergence and noise robustness, respectively. To further handle the false negative in PL and NL, NEVER proposes to smoothen and sharpen the training targets with the predictions from a twin momentum model. Extensive experiments on the various V+L tasks verify the effectiveness of the proposed method. The code will be released upon acceptance.

Cross-Modal Retrieval With Noisy Correspondence via Consistency Refining and Mining.

The success of existing cross-modal retrieval (CMR) methods heavily rely on the assumption that the annotated cross-modal correspondence is faultless. In practice, however, the correspondence of some pairs would be inevitably contaminated during data collection or annotation, thus leading to the so-called Noisy Correspondence (NC) problem. To alleviate the influence of NC, we propose a novel method termed Consistency REfining And Mining (CREAM) by revealing and exploiting the difference between correspondence and consistency. Specifically, the correspondence and the consistency only be coincident for true positive and true negative pairs, while being distinct for false positive and false negative pairs. Based on the observation, CREAM employs a collaborative learning paradigm to detect and rectify the correspondence of positives, and a negative mining approach to explore and utilize the consistency. Thanks to the consistency refining and mining strategy of CREAM, the overfitting on the false positives could be prevented and the consistency rooted in the false negatives could be exploited, thus leading to a robust CMR method. Extensive experiments verify the effectiveness of our method on three image-text benchmarks including Flickr30K, MS-COCO, and Conceptual Captions. Furthermore, we adopt our method into the graph matching task and the results demonstrate the robustness of our method against fine-grained NC problem. The code is available on https://github.com/XLearning-SCU/2024-TIP-CREAM.

Self-Constructing Stereo Correspondences for Unsupervised Multi-View Stereo

Self-Constructing Stereo Correspondences for Unsupervised Multi-View Stereo

Collaboration of Progressive Web App (PWA) And Firebase Cloud Messaging (FCM) for Optimal Performance Mailing Software

The problem of correspondence cannot be separated from the ease, accuracy and speed of the processing process. In research from Riswandi Ishak and H. Trizaka, they proposed report management software as well as notification of disposition and monitoring of correspondence as a solution to correspondence problems. However, there are shortcomings in the notification process for the disposition of correspondence, which requires the software to be actively open. In research, P. Dwi proposed a notification solution with Firebase Cloud Messaging (FCM), so that it can send notifications as long as the browser is active and connected to the internet, even without opening the software. There is a problem currently when the software becomes unstable when the internet connection is bad or offline. Progressive Web Apps (PWA) offers the concept of web-based application development by implementing browser technology such as service workers and app manifests. PWA is capable of displaying pages offline but cannot save, change, or delete data in the database. The test results of this research used Lighthouse and showed an average score of 100 on the PWA criteria, 85 on the performance criteria, 97 on the accessibility criteria, and 100 on the best practices criteria. Additional results obtained by implementing PWA mean page loading times are 26.6% faster with cache and service workers. The PWA and FCM concepts provide the best experience in using Mailing Software even with minimal internet connection or offline. This strategy was chosen to still get a fast response when running the mail processing software.

Composer's video for his own work: time parallels, dotted lines, perpendiculars

Relevance of the study is related to the problem of composers' self-illustrations in the YouTube presentations. The novelty of this topic is conditioned by new practice to illustrate music in YouTube. Graphic representation of music leads to the problem of correspondence between music and video. We selected only two Ukrainian composers to compare: Yuri Ishchenko and Sviatoslav Luniov.
 The main objective of the study is the determination of main principles of similarity or dissimilarity of time structures in music and images. The methodology is based on methods of comparison and analogy in the context of phenomenological approach to the Event in the Being Time (by late Heidegger). How the study was done: time structures of selected compositions were compared, through analogies, with the illustrating graphic events as phenomena of existential time. Thereby a number of similarities and dissimilarities were detected.
 Composer Y. Ishchenko preferred to illustrate his music by static images. Such a situation is a paradoxical because of the dynamic essence of his music. It establishes the problem of co-existent two time situations: in the first of them there is an integral and quasi-timeless image of music; but in the other situation there is a running stream of music structures. This approach becomes very and very fruitful because Music is able to use both the continuity, integrity of musical image as well as fluid sound events. The Y. Ishchenko's "Watercolors" for violin and piano are illustrated by several images; this approach makes it possible to mark different parts of total design (i.e. different movements or stages). Another solution one can hear and see in his "Small partita" for violin and harp. I helped him to accompany this design with fragments of animation about baroque époque. There are several GIFs - the periodically repeating images, which creates a cross-rhythm constellation with music. The significant crossfade is present in the video by Ishchenko's 3rd Violin sonata: ruined house through cross-like gap transforms in an abandoned church. The final fade of it lasts a minute. Thus, the minimal graphic processes are extremely expressive. S. Luniov mastered advanced skills in electronic music and its visual presentation. In his video "POST” he combines fragments of his own Panta Rhei for orchestra with sounds of the war. The intensity of the music development is essentially higher than that of graphic illustration, but at the end of the project the situation is opposite. In the video accompanying his Panta Rhei for orchestra S. Luniov synchronized main events and stages of the sound score with more intensive changes of the video stream and its thematic.
 Findings and conclusions. For the composer Y. Ishchenko, the graphic self-illustration on YouTube was mostly the outer necessity. He often limited his projects with static images for the total score or its main parts. Thus, he reinforced the integrity pole of events with the graphic ingredient of YouTube projects. Thereby, the dramaturgic tense of his compositions is concentrated in music. Such an approach was called time perpendicular strategy or time dotted line strategy. For the composer S. Luniov, the graphic self-illustration on YouTube is mostly the parallel stream of events. However, the tempo and intensity of visual and sound streams can be unequal. Such an approach is called time parallel strategy. Significance of the study follows from its conclusions, leading to more skillful and exact solutions within composers' self-illustration, i.e. in clarifying the semantic and structure design in music.