Depth Of Object Research Articles

Object Depth and Size Estimation Using Stereo-Vision and Integration With SLAM

Object Depth and Size Estimation Using Stereo-Vision and Integration With SLAM

The nature of compact radio-loud AGN: a systematic look at the LOFAR AGN population

ABSTRACT We investigate the nature of low-luminosity radio-loud active galactic nuclei (RLAGN) selected from the LOFAR Two-metre Sky Survey (LoTSS) first data release (DR1). Using optical, mid-infrared, and radio data, we have conservatively selected 55 radiative AGN candidates from DR1 within the redshift range 0.03 < z < 0.1. We show using high-frequency Karl G. Jansky Very Large Array (VLA) observations that 10 out of 55 objects show radio emission on scales >1–3 kpc, 42 are compact at the limiting resolution of 0.35 arcsec (taking an upper limit on the projected physical size, this corresponds to less than 1 kpc), and three are undetected. The extended objects display a wide range of radio morphologies: two-jet (5), one-jet (4), and double-lobed (1). We present the radio spectra of all detected radio sources which range from steep to flat/inverted and span the range seen for other compact radio sources such as compact symmetric objects (CSOs), compact steep spectrum (CSS) sources, and gigahertz peaked-spectrum (GPS) sources. Assuming synchrotron self-absorption (SSA) for flat/inverted radio spectrum sources, we predict small physical sizes for compact objects to range between 2 and 53 pc. Alternatively, using free–free absorption (FFA) models, we have estimated the free electron column depth for all compact objects, assuming a homogeneous absorber. We find that these objects do not occupy a special position on the power/linear size (P − D) diagram but some share a region with radio-quiet quasars (RQQs) and the so-called ‘FR0’ sources in terms of radio luminosity and linear size.

Optical lens optimization design and tolerance analysis for a depth camera

This paper describes a depth camera lens design composed of eight glass lenses, where the field of view is 100°, the focal length is 1.88 mm, and the F/# is 2. The lens design requires the lens to be fixed and the object distance to be fixed at −745.964 mm. The depth of focus ranges from −0.0047 mm to 0.0096 mm and can be obtained by selecting a spatial frequency of 60 lp/mm for the Through-focus MTF optimization design. The object distance ranges from negative infinity (-∞) to −250 mm (for the depth of field), the MTF is greater than 0.73 at 60 lp/mm, the lateral color difference is less than 1.58 μm, the optical distortion is less than 1%, and the relative illumination is greater than 64%. When conducting tolerance analysis, we select a spatial frequency requirement of tolerance MTF>0.3. At an object distance of −745.964 mm, the lens resolution can reach 80 lp/mm. The object distance is -∞, and the lens resolution is above 59 lp/mm. The object distance is −250 mm, and the lens resolution can reach over 43 lp/mm. In this work, the depth camera lens is designed to have an object distance shooting range of −6 m to −0.25 m. When the object distance is −6 m, the object depth resolution is 61 mm; while at the object distance of −0.25 m, the object depth resolution is 0.1045 mm.

Buried object localization by spectral analysis of surface wave reflections

A proposed method of buried object localization, based upon spectral analysis of surface wave reflections, is investigated numerically. This arises from a hypothesis to the effect that reflectivity would be maximized at some intermediate wavelength, shorter ones associated with disturbances too shallow to substantially excite the object as a secondary (reflecting) source, and longer ones with disturbances involving such volumes of earth in motion that the object become insignificant as a reflector. Discernment of that intermediate wavelength of maximum reflectivity might, thereby, provide an index of the object’s depth. Horizontal distance from echo return time, and azimuth from phase array techniques, could complete a localization methodology. Presented, accordingly, are 2D FEA simulations, in which narrow-banded surface wave trains are excited by a point source on the surface of an elastic medium, a void provided at some horizontal distance and depth, the strength of reflected surface motions thereafter examined in relation to frequency. These bear out the hypothesis, finding maximal reflectivity for a surface wavelength about two-thirds of the object depth. Proposed is that this could serve as the basis for a horizontal stand-off detection method, especially for applications like landmines, for which it may be undesirable to scan from above.

Stereo vision based object detection for autonomous navigation in space environments

Obstacle detection and avoidance are the major issues in autonomous navigation for partially known or unknown environments. With the proliferation of space debris, researchers are actively investigating debris removal to facilitate future space operations. This calls for the development of autonomous navigation techniques for space missions. Free-space object detection is a crucial task in intelligent systems, particularly for path planning. In this study, we propose a stereo vision-based intelligent system for space object detection, using two vertically aligned omnidirectional stereo cameras separated by 10 cm. Firstly, a single-shot multibox detector (SSD) based on deep learning is employed to identify the objects present in the image. Then, the triangulation method is used to determine the distance between the object and the system. The proposed system can provide object depth information up to a maximum range of 1.1 km in a space environment.

Deep Multi-task Learning for Animal Chest Circumference Estimation from Monocular Images

The applications of deep learning algorithms with images to various scenarios have attracted significant research attention. However, application scenarios in animal breeding managements are still limited. In this paper we propose a new deep learning framework to estimate the chest circumference of domestic animals from images. This parameter is a key metric for breeding and monitoring the quality of animal in animal husbandry. We design a set of feature extraction methods based on a multi-task learning framework to address the challenging issues in the main estimation task. The multiple tasks in our proposed framework include object segmentation, keypoint estimation, and depth estimation of cow from monocular images. The domain-specific features extracted from these tasks improve upon our main estimation task. In addition, we also attempt to reduce unnecessary computations during the framework design to reduce the cost of subsequent practical implementation of the developed system. Our proposed framework is tested on our own collected dataset to evaluate its performance.

Thin and Large Depth-Of-Field Compound-Eye Imaging for Close-Up Photography

Large depth of field (DOF) and stereo photography are challenging yet rewarding areas of research in close-up photography. In this study, a compound-eye imaging system based on a discrete microlens array (MLA) was implemented for close-range thin imaging. A compact imaging system with a total length of 3.5 mm and a DOF of 7 mm was realized using two planar aspherical MLAs in a hexagonal arrangement. A new three-layer structure and discrete arrangement of sublenses were proposed to suppress stray light and enable the spatial refocusing method, which restores image information at different object depths. The system is successfully fabricated, and the system performance is carefully investigated. Our system offers a large depth of field, high resolution, and portability, making it ideal for close-up photography applications requiring a short conjugate distance and small device volume, while also addressing the issue of crosstalk between adjacent channels.

GRAMO: geometric resampling augmentation for monocular 3D object detection

Data augmentation is widely recognized as an effective means of bolstering model robustness. However, when applied to monocular 3D object detection, non-geometric image augmentation neglects the critical link between the image and physical space, resulting in the semantic collapse of the extended scene. To address this issue, we propose two geometric-level data augmentation operators named Geometric-Copy-Paste (Geo-CP) and Geometric-Crop-Shrink (Geo-CS). Both operators introduce geometric consistency based on the principle of perspective projection, complementing the options available for data augmentation in monocular 3D. Specifically, Geo-CP replicates local patches by reordering object depths to mitigate perspective occlusion conflicts, and Geo-CS re-crops local patches for simultaneous scaling of distance and scale to unify appearance and annotation. These operations ameliorate the problem of class imbalance in the monocular paradigm by increasing the quantity and distribution of geometrically consistent samples. Experiments demonstrate that our geometric-level augmentation operators effectively improve robustness and performance in the KITTI and Waymo monocular 3D detection benchmarks.

Robust multiple obstacle tracking method based on depth aware OCSORT for agricultural robots

Multiple Object Tracking (MOT) of dynamic obstacles in field is an important prerequisite for agricultural robots to achieve dynamic obstacle avoidance. The complex and unpredictable road environment in the rural areas will cause severe vibration to the robot, affecting the camera pose and consequently leading to object matching errors. Therefore, we propose an improved method named Depth Aware Observation Centric Simple Online and Realtime Tracking (DA-OCSORT) with two new modules named Inertial Measurement Unit (IMU)-based Camera Motion Compensation (ICMC) and Depth Aware (DA). This method can use IMU information to compensate for camera ego-motion and perform multi-dimensional matching through object depth information, thereby minimizing the influence of camera motion on tracking process. We created an open-source MOT dataset, named MOTorchard. Building upon the pure motion-based method OCSORT, we achieve SOTA on this MOTorchard with HOTA 54.1, MOTA 83.3, IDF1 52.9 and FPS 22.7. Moreover, the proposed two modules prove to have good real-time performance and strong adaptability to other published MOT algorithms.

Effects of Focal Distance on Near-Field Depth Perception and Accommodative Response in a VariFocal Optical See-Through Augmented Reality Display.

Through a human-subject experiment, we investigated the effects of focal distance on depth perception and accommodative response in an optical see-through augmented reality (AR) display. The display was able to provide focus cues and was rigorously calibrated. The near-field distances ranging between 3 diopters and 1 diopter were considered as target distance. In the experiment, it was found that the perceived depth of a virtual object was significantly biased along with the focal distance of virtual image plane of the display. In addition, the experimental results implied that the perceived depth of a virtual object would be potentially more accurate in the condition where the focal distance of virtual image plane was consistent with the target distance than in the conditions where it could deviate from the target distance. Regarding accommodative response, it was found that the response to a virtual object changed along with the focal distance of virtual image plane as well as the target distance. However, the changing rate depending on target distance was less steep in the conditions where the focal distance could be mismatched with the target distance than in the condition where it was consistent with the target distance. In the consistent condition, the changing rate of accommodative responses to virtual objects were similar to that for their physical counterparts.

Expediting Large-Scale Vision Transformer for Dense Prediction Without Fine-Tuning.

In a wide range of dense prediction tasks, large-scale Vision Transformers have achieved state-of-the-art performance while requiring expensive computation. In contrast to most existing approaches accelerating Vision Transformers for image classification, we focus on accelerating Vision Transformers for dense prediction without any fine-tuning. We present two non-parametric operators specialized for dense prediction tasks, a token clustering layer to decrease the number of tokens for expediting and a token reconstruction layer to increase the number of tokens for recovering high-resolution. To accomplish this, the following steps are taken: i) token clustering layer is employed to cluster the neighboring tokens and yield low-resolution representations with spatial structures; ii) the following transformer layers are performed only to these clustered low-resolution tokens; and iii) reconstruction of high-resolution representations from refined low-resolution representations is accomplished using token reconstruction layer. The proposed approach shows promising results consistently on 6 dense prediction tasks, including object detection, semantic segmentation, panoptic segmentation, instance segmentation, depth estimation, and video instance segmentation. Additionally, we validate the effectiveness of the proposed approach on the very recent state-of-the-art open-vocabulary recognition methods. Furthermore, a number of recent representative approaches are benchmarked and compared on dense prediction tasks.

USOD10K: A New Benchmark Dataset for Underwater Salient Object Detection.

Underwater salient object detection (USOD) attracts increasing interest for its promising performance in various underwater visual tasks. However, USOD research is still in its early stages due to the lack of large-scale datasets within which salient objects are well-defined and pixel-wise annotated. To address this issue, this paper introduces a new dataset named USOD10K. It consists of 10,255 underwater images, covering 70 categories of salient objects in 12 different underwater scenes. In addition, salient object boundaries and depth maps of all images are provided in this dataset. The USOD10K is the first large-scale dataset in the USOD community, making a significant leap in diversity, complexity, and scalability. Secondly, a simple but strong baseline termed TC-USOD is designed for the USOD10K. The TC-USOD adopts a hybrid architecture based on an encoder-decoder design that leverages transformer and convolution as the basic computational building block of the encoder and decoder, respectively. Thirdly, we make a comprehensive summarization of 35 cutting-edge SOD/USOD methods and benchmark them over the existing USOD dataset and the USOD10K. The results show that our TC-USOD obtained superior performance on all datasets tested. Finally, several other use cases of the USOD10K are discussed, and future directions of USOD research are pointed out. This work will promote the development of the USOD research and facilitate further research on underwater visual tasks and visually-guided underwater robots. To pave the road in this research field, all the dataset, code, and benchmark results are publicly available: https://github.com/LinHong-HIT/USOD10K.

Cylin-Painting: Seamless 360° Panoramic Image Outpainting and Beyond.

Image outpainting gains increasing attention since it can generate the complete scene from a partial view, providing a valuable solution to construct 360° panoramic images. As image outpainting suffers from the intrinsic issue of unidirectional completion flow, previous methods convert the original problem into inpainting, which allows a bidirectional flow. However, we find that inpainting has its own limitations and is inferior to outpainting in certain situations. The question of how they may be combined for the best of both has as yet remained under-explored. In this paper, we provide a deep analysis of the differences between inpainting and outpainting, which essentially depends on how the source pixels contribute to the unknown regions under different spatial arrangements. Motivated by this analysis, we present a Cylin-Painting framework that involves meaningful collaborations between inpainting and outpainting and efficiently fuses the different arrangements, with a view to leveraging their complementary benefits on a seamless cylinder. Nevertheless, straightforwardly applying the cylinder-style convolution often generates visually unpleasing results as it discards important positional information. To address this issue, we further present a learnable positional embedding strategy to incorporate the missing component of positional encoding into the cylinder convolution, which significantly improves the panoramic results. It is noted that while developed for image outpainting, the proposed algorithm can be effectively extended to other panoramic vision tasks, such as object detection, depth estimation, and image super-resolution. Code will be made available at https://github.com/KangLiao929/Cylin-Painting.

GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection.

Geometry plays a significant role in monocular 3D object detection. It can be used to estimate object depth by using the perspective projection between object's physical size and 2D projection in the image plane, which can introduce mathematical priors into deep models. However, this projection process also introduces error amplification, where the error of the estimated height is amplified and reflected into the projected depth. It leads to unreliable depth inferences and also impairs training stability. To tackle this problem, we propose a novel Geometry Uncertainty Propagation Network (GUPNet++) by modeling geometry projection in a probabilistic manner. This ensures depth predictions are well-bounded and associated with a reasonable uncertainty. The significance of introducing such geometric uncertainty is twofold: (1). It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning. (2). It can be derived to a highly reliable confidence to indicate the quality of the 3D detection result, enabling more reliable detection inference. Experiments show that the proposed approach not only obtains (state-of-the-art) SOTA performance in image-based monocular 3D detection but also demonstrates superiority in efficacy with a simplified framework.

Transparent Object Depth Perception Network for Robotic Manipulation Based on Orientation-Aware Guidance and Texture Enhancement

Transparent Object Depth Perception Network for Robotic Manipulation Based on Orientation-Aware Guidance and Texture Enhancement

Self-Supervised Monocular Depth Estimation With Positional Shift Depth Variance and Adaptive Disparity Quantization.

Recently, attempts to learn the underlying 3D structures of a scene from monocular videos in a fully self-supervised fashion have drawn much attention. One of the most challenging aspects of this task is to handle independently moving objects as they break the rigid-scene assumption. In this paper, we show for the first time that pixel positional information can be exploited to learn SVDE (Single View Depth Estimation) from videos. The proposed moving object (MO) masks, which are induced by the depth variance to shifted positional information (SPI) and are referred to as 'SPIMO' masks, are highly robust and consistently remove independently moving objects from the scenes, allowing for robust and consistent learning of SVDE from videos. Additionally, we introduce a new adaptive quantization scheme that assigns the best per-pixel quantization curve for depth discretization, improving the fine granularity and accuracy of the final aggregated depth maps. Finally, we employ existing boosting techniques in a new way that self-supervises moving object depths further. With these features, our pipeline is robust against moving objects and generalizes well to high-resolution images, even when trained with small patches, yielding state-of-the-art (SOTA) results with four- to eight-fold fewer parameters than the previous SOTA techniques that learn from videos. We present extensive experiments on KITTI and CityScapes that show the effectiveness of our method.

Application of Simple Refraction Correction Method for Shallow Coastal Bathymetric Mapping Based on UAV-Photogrammetry

Shallow coastal bathymetric data has an important role in various applications, especially in maritime and coastal management fields. Various survey techniques to produce such data have been carried out by several researchers, one technique that is cheap, covers the wide area, flexible, and produces high-resolution bathymetric data is UAV-photogrammetry. However, this technique is affected by refraction effects that cause the estimated depth of underwater objects to be shallower than fact, thus reducing the accuracy of the bathymetric model. Therefore, the objective of this study is to present a simple refraction correction method based on the Least-Square method. To test the reliability of the method, the accuracy of the model is compared with two other methods (without correction and using a correction factor = 1.34). In addition, the effectiveness of the method was also tested through its application in various survey conditions. Overall, the proposed method outperforms the two existing methods. It is also very effective in reducing the error value during high tide conditions by up to 70%. The height of the UAV does not significantly affect the accuracy of the correction model, so in this case it is recommended to use an altitude of 100 m for survey efficiency.

Monocular 3D object detection for construction scene analysis

AbstractThree‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

3D measurement using fringe projection profilometry

This work is devoted to measuring the depth of the 3D object using the structured light method, in particular, phase shift profilometry. Theoretical studies on the methods of three-dimensional measurement systems and fringe projection profilometry are presented. The phase shift profilometry method with an improved calculation of the frequency of sinusoidal patterns is applied. In practical implementation in the environment (20 cm × 30 cm), the algorithm is tested on a stepped object consisting of eight steps with a difference of 150 mm between two successive steps. In this case, the achievable error for measuring such an object is 20 mm. Our method has great potential in industrial applications where the measurement of the smoothing of the surface of the object is needed to find the defect in the surface with high accuracy without contacting the object.

Структура и эволюция углеводородных систем окраинных морей Северного Ледовитого океана (Восточная Арктика)

The article presents the results of studying the structure and evolution of hydrocarbon systems in the marginal seas of the Eastern Arctic — the Chukchi, East Siberian and Laptev Seas. The authors assess the hydrocarbon potential of the studied water areas basing on the results of the basin analysis and numerical modeling. Three promising sedimentary complexes Aptian–Upper Cretaceous, Paleocene–Eocene, and Oligocene–Lower Miocene are predicted in the composition of the sedimentary cover. It is shown that the development of hydrocarbon systems is determined mainly by the rate of sinking of the basins and the thickness of the formed overburden complexes. The main centers of hydrocarbon generation of the Aptian–Upper Cretaceous perspective complex are located in the North Chukchi trough, while those of the Paleocene–Eocene and Oligocene–Lower Miocene complex are located in the Laptev Sea basin. The most likely areas of hydrocarbon accumulation in the reservoirs of the Aptian–Upper Cretaceous complex are the marginal parts of the North Chukchi and East Siberian basins, where deposits are expected at a depth of ~5 km. In the Laptev Sea basin, in addition to the marginal zones, there is a large accumulation area in its central part with deposits predicted at a depth of ≥5 km. In the Paleogene (Paleocene–Eocene) complex, hydrocarbon accumulations are predicted mainly in the central parts of the studied basins and, to a lesser extent, in the near-side parts. The occurrence depths of promising objects vary from 5–6 km in the central parts to 2–3 km in the near-side ones. In the Oligocene-Lower Miocene complex, hydrocarbon accumulations are expected mainly within the Laptev Sea basin at depths of 3–5 km. Significant hydrocarbon potential may be associated with Paleogene clinoform deposits.