Correspondence Problem Research Articles

A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video

The utilization of stationary underwater cameras is a modern and well-adapted approach to provide a continuous and cost-effective long-term solution to monitor underwater habitats of particular interest. A common goal of such monitoring systems is to gain better insight into the dynamics and condition of populations of various marine organisms, such as migratory or commercially relevant fish taxa. This paper describes a complete processing pipeline to automatically determine the abundance, type and estimate the size of biological taxa from stereoscopic video data captured by the stereo camera of a stationary Underwater Fish Observatory (UFO). A calibration of the recording system was carried out in situ and, afterward, validated using the synchronously recorded sonar data. The video data were recorded continuously for nearly one year in the Kiel Fjord, an inlet of the Baltic Sea in northern Germany. It shows underwater organisms in their natural behavior, as passive low-light cameras were used instead of active lighting to dampen attraction effects and allow for the least invasive recording possible. The recorded raw data are pre-filtered by an adaptive background estimation to extract sequences with activity, which are then processed by a deep detection network, i.e., Yolov5. This provides the location and type of organisms detected in each video frame of both cameras, which are used to calculate stereo correspondences following a basic matching scheme. In a subsequent step, the size and distance of the depicted organisms are approximated using the corner coordinates of the matched bounding boxes. The Yolov5 model employed in this study was trained on a novel dataset comprising 73,144 images and 92,899 bounding box annotations for 10 categories of marine animals. The model achieved a mean detection accuracy of 92.4%, a mean average precision (mAP) of 94.8% and an F1 score of 93%.

Feature-Based Occupancy Map-Merging for Collaborative SLAM.

One of the most frequently used approaches to represent collaborative mapping are probabilistic occupancy grid maps. These maps can be exchanged and integrated among robots to reduce the overall exploration time, which is the main advantage of the collaborative systems. Such map fusion requires solving the unknown initial correspondence problem. This article presents an effective feature-based map fusion approach that includes processing the spatial occupancy probabilities and detecting features based on locally adaptive nonlinear diffusion filtering. We also present a procedure to verify and accept the correct transformation to avoid ambiguous map merging. Further, a global grid fusion strategy based on the Bayesian inference, which is independent of the order of merging, is also provided. It is shown that the presented method is suitable for identifying geometrically consistent features across various mapping conditions, such as low overlapping and different grid resolutions. We also present the results based on hierarchical map fusion to merge six individual maps at once in order to constrict a consistent global map for SLAM.

Synthetic Undecidability and Incompleteness of First-Order Axiom Systems in Coq

We mechanise the undecidability of various first-order axiom systems in Coq, employing the synthetic approach to computability underlying the growing Coq Library of Undecidability Proofs. Concretely, we cover both semantic and deductive entailment in fragments of Peano arithmetic (PA) as well as ZF and related finitary set theories, with their undecidability established by many-one reductions from solvability of Diophantine equations, i.e. Hilbert’s tenth problem (H10), and the Post correspondence problem (PCP), respectively. In the synthetic setting based on the computability of all functions definable in a constructive foundation, such as Coq’s type theory, it suffices to define these reductions as meta-level functions with no need for further encoding in a formalised model of computation. The concrete cases of PA and the considered set theories are supplemented by a general synthetic theory of undecidable axiomatisations, focusing on well-known connections to consistency and incompleteness. Specifically, our reductions rely on the existence of standard models, necessitating additional assumptions in the case of full ZF, and all axiomatic extensions still justified by such standard models are shown incomplete. As a by-product of the undecidability of set theories formulated using only membership and no equality symbol, we obtain the undecidability of first-order logic with a single binary relation.

Самодостатність територіальних громад в умовах реформи місцевого самоврядування: невирішені проблеми та шляхи їх вирішення

The reform of local self-government in Ukraine led to the expansion of the powers of territorial communities and an increase in their financial resources. There was a problem of correspondence of powers and financial resources. The methodology for assessing the capacity of territorial communities was not developed during the years of the decentralization reform. The article proposes a set of measures aimed at increasing the self-sufficiency of territorial communities in terms of determining their powers and size, improving the mechanism of horizontal budget equalization, redistribution of taxes between the central government and local self-government, taking into account European experience, increasing the share of non-targeted transfers, developing standards for the provision of public services.

A new stage of humanization of the criminal-legal impact regarding the application of punishment in the type of life imprisonment: the problem of correspondence.

The scientific article explores a new stage of humanization of the criminal-legal influence on the application of punishment in the form of life imprisonment. It has been established that a number of decisions of the European Court of Human Rights served as a prerequisite for this process, which generally stated the absence of a mechanism for early release of convicts from serving a sentence of life imprisonment in Ukraine, which would allow the educational purpose of this punishment to be realized. Attention is focused on the fact that the European Court of Human Rights does not oblige the state to provide specific forms of early release from serving a sentence of life imprisonment. At the same time, it is emphasized that this process should be legally defined, transparent and provide the person sentenced to life imprisonment with the "right to hope". An inconsistency was established in the text of the motivational and operative parts of the decision of the Constitutional Court of Ukraine dated September 16, 2021 No. 6-р (II)/2021 (case on review of the sentence of a person sentenced to life imprisonment) in the part of determining the types of early release from serving a sentence in life imprisonment. In the decisive part of the decision, the Constitutional Court of Ukraine obliged the Verkhovna Rada of Ukraine to introduce two independent, independent measures of a criminal law nature - the replacement of the punishment in the form of life imprisonment with a milder punishment and conditional early release from serving the punishment in the form of life imprisonment will. The article focuses attention on the need to observe the reasonableness and justice of criminal law influence in the process of humanizing life imprisonment. It is argued that excessive humanization of life imprisonment may lead to a decrease in the protective function of criminal legislation and the ability to prevent new criminal offenses. It was concluded that in the conditions of martial law there is an increase in the number of crimes punishable by life imprisonment. The position is expressed that it should be recognized as socially determined to introduce the replacement of punishment in the form of life imprisonment with a milder one and subsequent conditional early release from serving the prescribed sentence, if the convicted person corrects himself and shows his readiness for further resocialization.

Multi-view triangulation without correspondences

We introduce a novel method to solve the joint problem of correspondence and triangulation of points from multiple calibrated perspective views and show its application to counting the number of leaves present on plants photographed from multiple angles in phenotyping facilities. Assuming there is a set of n calibrated views of an object, the input to our algorithm is a set of 2D points (e.g., leaf tips, fruits, flowers) detected in each of the n views, and the output is a set of 3D points corresponding to the 2D points when re-projected on views. Our approach is robust to noise and occlusion. In particular, it is not required for all points to be visible from one of the views, as our algorithm can infer that a point is occluded by reasoning on the 3D geometry of the scene. Our algorithm is suitable for many points (approx. thousands) reconstructed from a reduced set of views (up to 15). For example, our implementation finds the correspondence of 20 points captured by six cameras in about one second on consumer hardware. We evaluate the performance on synthetic data as well as real examples. We show that the accuracy of leaf counting from multi-view images is drastically improved by our algorithm.

Stereo Dense Scene Reconstruction and Accurate Localization for Learning-Based Navigation of Laparoscope in Minimally Invasive Surgery.

The computation of anatomical information and laparoscope position is a fundamental block of surgical navigation in Minimally Invasive Surgery (MIS). Recovering a dense 3D structure of surgical scene using visual cues remains a challenge, and the online laparoscopic tracking primarily relies on external sensors, which increases system complexity. Here, we propose a learning-driven framework, in which an image-guided laparoscopic localization with 3D reconstructions of anatomical structures is obtained. To reconstruct the structure of the whole surgical environment, we first fine-tune a learning-based stereoscopic depth perception method, which is robust to texture-less and variant soft tissues, for depth estimation. Then, we develop a dense reconstruction algorithm to represent the scene by surfels, estimate the laparoscope poses and fuse the depth into a unified reference coordinate for tissue reconstruction. To estimate poses of new laparoscope views, we achieve a coarse-to-fine localization method, which incorporates our reconstructed model. We evaluate the reconstruction method and the localization module on three datasets, namely, the stereo correspondence and reconstruction of endoscopic data (SCARED), the ex-vivo data collected with Universal Robot (UR) and Karl Storz Laparoscope, and the in-vivo DaVinci robotic surgery dataset, where the reconstructed structures have rich details of surface texture with an error under 1.71 mm and the localization module can accurately track the laparoscope with images as input. Experimental results demonstrate the superior performance of the proposed method in anatomy reconstruction and laparoscopic localization. The proposed framework can be potentially extended to the current surgical navigation system.

BuildMapper: A fully learnable framework for vectorized building contour extraction

Deep learning based methods have significantly boosted the study of automatic building extraction from remote sensing images. However, delineating vectorized and regular building contours like a human does remains very challenging, due to the difficulty of the methodology, the diversity of building structures, and the imperfect imaging conditions. In this paper, we propose the first end-to-end learnable building contour extraction framework, named BuildMapper, which can directly and efficiently delineate building polygons just as a human does. BuildMapper consists of two main components: 1) a contour initialization module that generates initial building contours; and 2) a contour evolution module that performs both contour vertex deformation and reduction, which removes the need for complex empirical post-processing used in existing methods. In both components, we provide new ideas, including a learnable contour initialization method to replace the empirical methods, dynamic predicted and ground truth vertex pairing for the static vertex correspondence problem, and a lightweight encoder for vertex information extraction and aggregation, which benefit a general contour-based method; and a well-designed vertex classification head for building corner vertices detection, which casts light on direct structured building contour extraction. We also built a suitable large-scale building dataset, the WHU-Mix (vector) building dataset, to benefit the study of contour-based building extraction methods. The extensive experiments conducted on the WHU-Mix (vector) dataset, the WHU dataset, and the CrowdAI dataset verified that BuildMapper can achieve a state-of-the-art performance, with a higher mask average precision (AP) and boundary AP than both segmentation-based and contour-based methods. We also confirmed that more than 60.0/50.8% of the building polygons predicted by BuildMapper in the WHU-Mix (vector) test sets I/II, 84.2% in the WHU building test set, and 68.3% in the CrowdAI test set are on par with the manual delineation level.

Simulating vocal learning of spoken language: Beyond imitation

Computational approaches have an important role to play in understanding the complex process of speech acquisition, in general, and have recently been popular in studies of vocal learning in particular. In this article we suggest that two significant problems associated with imitative vocal learning of spoken language, the speaker normalisation and phonological correspondence problems, can be addressed by linguistically grounded auditory perception. In particular, we show how the articulation of consonant–vowel syllables may be learnt from auditory percepts that can represent either individual utterances by speakers with different vocal tract characteristics or ideal phonetic realisations. The result is an optimisation-based implementation of vocal exploration – incorporating semantic, auditory, and articulatory signals – that can serve as a basis for simulating vocal learning beyond imitation.

Dual Projective Zero-Shot Learning Using Text Descriptions

Zero-shot learning (ZSL) aims to recognize image instances of unseen classes solely based on the semantic descriptions of the unseen classes. In this field, Generalized Zero-Shot Learning (GZSL) is a challenging problem in which the images of both seen and unseen classes are mixed in the testing phase of learning. Existing methods formulate GZSL as a semantic-visual correspondence problem and apply generative models such as Generative Adversarial Networks and Variational Autoencoders to solve the problem. However, these methods suffer from the bias problem since the images of unseen classes are often misclassified into seen classes. In this work, a novel model named the Dual Projective model for Zero-Shot Learning (DPZSL) is proposed using text descriptions. In order to alleviate the bias problem, we leverage two autoencoders to project the visual and semantic features into a latent space and evaluate the embeddings by a visual-semantic correspondence loss function. An additional novel classifier is also introduced to ensure the discriminability of the embedded features. Our method focuses on a more challenging inductive ZSL setting in which only the labeled data from seen classes are used in the training phase. The experimental results, obtained from two popular datasets—Caltech-UCSD Birds-200-2011 (CUB) and North America Birds (NAB)—show that the proposed DPZSL model significantly outperforms both the inductive ZSL and GZSL settings. Particularly in the GZSL setting, our model yields an improvement up to 15.2% in comparison with state-of-the-art CANZSL on datasets CUB and NAB with two splittings.

Steformer: Efficient Stereo Image Super-Resolution With Transformer

With the rapid development of stereoscopic vision applications, stereo image processing techniques have attracted increasing attention in both academic and industrial communities. In this paper, we study the fundamental stereo image super-resolution (SR) problem, which aims to recover high-resolution stereo images from low-resolution (LR) stereo images. Since disparities between stereo images vary significantly, convolutional network-based stereo image SR methods show a limitation in capturing long-range dependencies. To address this problem, this paper proposes to leverage the capability of self-attention in Transformers to efficiently capture reliable stereo correspondence and incorporate cross-view information for stereo image SR. Our model, named Steformer, consists of three parts: cross attentive feature extraction, cross-to-intra information integration and high-quality image reconstruction. In particular, the cross attentive feature extraction module employs residual cross Steformer blocks (RCSB) for long-range cross-view information extraction. Then, the cross-to-intra information integration module exploits cross-view and intra-view information using cross-to-intra attention mechanism (C2IAM). Finally, residual Steformer blocks (RSB) are designed for feature pre-processing in high-quality image reconstruction. Extensive experiments show that Steformer achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations, while the total number of parameters can be reduced by up to 40.71%.

StereoPairFree: Self-Constructed Stereo Correspondence Network From Natural Images

This article proposes a variant of a convolutional neural network-based stereo matching method, StereoPairFree, that requires only normal and not left or right stereo images. The corresponding image patches for training stereo matching cost networks look similar, apart from some properties, such as different illumination, occlusion, distortion, and foreshortening. We propose a method for generating synthesized pairs of corresponding image patches for a given image patch. We also propose a multipatch matching cost network that exploits various input patch sizes. The proposed matching cost network is optimized by cross-based cost aggregation and semiglobal matching, followed by consistency checks and bilateral filtering. Hence, StereoPairFree does not require a single stereo pair but standard images to build a deep learning stereo matching method. It is the first stereo matching method constructed from normal images and significantly outperforms monocular stereo matching approaches (Hur and Roth, 2020), (Hur and Roth, 2021), (Schuster et al., 2020). We evaluated StereoPairFree with the KITTI-2012, KITTI-2015, and Middlebury datasets. It significantly outperformed the baseline and several deep learning-based methods using these datasets.

The boundary behavior of $\mathcal Q_{p,q}$-homeomorphisms

This article studies systematically the boundary correspondence problem for $\mathcal Q_{p,q}$-homeomorphisms. The presented example demonstrates a deformation of the Euclidean boundary with the weight function degenerating on the boundary.

Robust Multi-View Clustering With Incomplete Information.

The success of existing multi-view clustering methods heavily relies on the assumption of view consistency and instance completeness, referred to as the complete information. However, these two assumptions would be inevitably violated in data collection and transmission, thus leading to the so-called Partially View-unaligned Problem (PVP) and Partially Sample-missing Problem (PSP). To overcome such incomplete information challenges, we propose a novel method, termed robuSt mUlti-view clusteRing with incomplEte information (SURE), which solves PVP and PSP under a unified framework. In brief, SURE is a novel contrastive learning paradigm which uses the available pairs as positives and randomly chooses some cross-view samples as negatives. To reduce the influence of the false negatives caused by random sampling, SURE is with a noise-robust contrastive loss that theoretically and empirically mitigates or even eliminates the influence of the false negatives. To the best of our knowledge, this could be the first successful attempt that simultaneously handles PVP and PSP using a unified solution. In addition, this could be one of the first studies on the noisy correspondence problem (i.e., the false negatives) which is a novel paradigm of noisy labels. Extensive experiments demonstrate the effectiveness and efficiency of SURE comparing with 10 state-of-the-art approaches on the multi-view clustering task.

The boundary behavior of $\mathcal Q_{p,q}$-homeomorphisms

This article studies systematically the boundary correspondence problem for $\mathcal Q_{p,q}$-homeomorphisms. The presented example demonstrates a deformation of the Euclidean boundary with the weight function degenerating on the boundary. Bibliography: 72 titles.

Mathematical Morphology View of Topological Rough Sets and Its Applications

This article focuses on the relationship between mathematical morphology operations and rough sets, mainly based on the context of image retrieval and the basic image correspondence problem. Mathematical morphological procedures and set approximations in rough set theory have some clear parallels. Numerous initiatives have been made to connect rough sets with mathematical morphology. Numerous significant publications have been written in this field. Others attempt to show a direct connection between mathematical morphology and rough sets through relations, a pair of dual operations, and neighborhood systems. Rough sets are used to suggest a strategy to approximate mathematical morphology within the general paradigm of soft computing. A single framework is defined using a different technique that incorporates the key ideas of both rough sets and mathematical morphology. This paper examines rough set theory from the viewpoint of mathematical morphology to derive rough forms of the morphological structures of dilation, erosion, opening, and closing. These newly defined structures are applied to develop algorithm for the differential analysis of chest X-ray images from a COVID-19 patient with acute pneumonia and a health subject. The algorithm and rough morphological operations show promise for the delineation of lung occlusion in COVID-19 patients from chest X-rays. The foundations of mathematical morphology are covered in this article. After that, rough set theory ideas are taken into account, and their connections are examined. Finally, a suggested image retrieval application of the concepts from these two fields is provided.

On bi-infinite and conjugate post correspondence problems

We study two modifications of the Post Correspondence Problem (PCP), namely (1) the bi-infinite version, where it is asked whether there exists a bi-infinite word such that two given morphisms agree on it, and (2) the conjugate version, where we require the images of a solution for two given morphisms are conjugates of each other. For the conjugate PCP we give an undecidability proof by reducing it to the word problem for a special type of semi-Thue systems and for the bi-infinite PCP we give a simple argument that it is in the class Σ20 of the arithmetical hierarchy.

Stereopsis without correspondence.

Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Integer Weighted Automata on Infinite Words

In this paper we combine two classical generalisations of finite automata (weighted automata and automata on infinite words) into a model of integer weighted automata on infinite words and study the universality and the emptiness problems under zero weight acceptance. We show that the universality problem is undecidable for three-state automata by a direct reduction from the infinite Post correspondence problem. We also consider other more general acceptance conditions as well as their complements with respect to the universality and the emptiness problems. Additionally, we build a universal integer weighted automaton with fixed transitions. This automaton has an additional integer input that allows it to simulate any semi-Thue system.

3D Vision Using Multiple Structured Light-Based Kinect Depth Cameras

Real-time 3D scanning of a scene or object using multiple depth cameras is often required in many applications but is still a challenging task for the computer vision community, especially when the object or scene is partially occluded and dynamic. If active depth sensors are used in this case, their resulting depth map quality gets degraded due to interference between active radiations from each depth sensor. Passive 3D sensors like stereo cameras can avoid the issue of interference as they do not emit any active radiation, but they face correspondence problems. Since releasing the commodity depth sensor Microsoft Kinect, researchers are getting more interested in active depth-sensing. However, Kinect sensors have some easily noticeable limitations concerning 3D reconstruction such as: they can provide depth maps for a limited range, their field of view is restricted and holes are observed in the depth map due to occlusion. The above-mentioned limitations can be overcome if multiple Kinect sensors are used simultaneously instead of a single Kinect sensor. Still, the challenge here is to avoid interference between these sensors. We present a comprehensive review of possible solutions to avoid interference between multiple Kinect sensors. Furthermore, we introduce the Kinect technology in detail along with applications where multiple Kinect sensors are used in the literature. We expect that this paper will be helpful to the researchers who want to use multiple Kinect sensors in sharing the workplace in their research.