Depth Images Research Articles

Research on the Anchor-Rod Recognition and Positioning Method of a Coal-Mine Roadway Based on Image Enhancement and Multiattention Mechanism Fusion-Improved YOLOv7 Model

A drill-anchor robot is an essential means of efficient drilling and anchoring in coal-mine roadways. It is significant to calculate the position of the drill-anchor robot based on the positioning information of the supported anchor rod to improve tunneling efficiency. Therefore, identifying and positioning the supported anchor rod has become a critical problem that needs to be solved urgently. Aiming at the problem that the target in the image is blurred and cannot be accurately identified due to the low and uneven illumination environment, we proposed an improved YOLOv7 (the seventh version of the You Only Look Once) model based on the fusion of image enhancement and multiattention mechanism, and the self-made dataset is used for testing and training. Aiming at the problem that the traditional positioning method cannot guarantee accuracy and efficiency simultaneously, an anchor-rod positioning method using depth image and RGB image alignment combined with least squares linear fitting is proposed, and the positioning accuracy is improved by processing the depth map. The results show that the improved model improves the mAP by 5.7% compared with YOLOv7 and can accurately identify the target. Through the positioning method proposed in this paper, the error between the positioning coordinate and the measurement coordinate of the target point on each axis does not exceed 11 mm, which has high positioning accuracy and improves the positioning accuracy and robustness of the anchor rod in the coal-mine roadway.

A Novel Method for Monocular Depth Estimation Using an Hourglass Neck Module.

In this paper, we propose a novel method for monocular depth estimation using the hourglass neck module. The proposed method has the following originality. First, feature maps are extracted from Swin Transformer V2 using a masked image modeling (MIM) pretrained model. Since Swin Transformer V2 has a different patch size for each attention stage, it is easier to extract local and global features from images input by the vision transformer (ViT)-based encoder. Second, to maintain the polymorphism and local inductive bias of the feature map extracted from Swin Transformer V2, a feature map is input into the hourglass neck module. Third, deformable attention can be used at the waist of the hourglass neck module to reduce the computation cost and highlight the locality of the feature map. Finally, the feature map traverses the neck and proceeds through a decoder, comprised of a deconvolution layer and an upsampling layer, to generate a depth image. To evaluate the objective reliability of the proposed method in this paper, we used the NYU Depth V2 dataset to compare and evaluate the methods published in other papers. As a result of the experiment, the RMSE value of the novel method for monocular depth estimation using the hourglass neck module proposed in this paper was 0.274, which was lower than those published in other papers. The lower the RMSE value, the better the depth estimation method; therefore, its efficiency compared to other techniques has been proven.

SpATr: MoCap 3D human action recognition based on spiral auto-encoder and transformer network

Recent technological advancements have significantly expanded the potential of human action recognition through harnessing the power of 3D data. This data provides a richer understanding of actions, including depth information that enables more accurate analysis of spatial and temporal characteristics. In this context, We study the challenge of 3D human action recognition. Unlike prior methods, that rely on sampling 2D depth images, skeleton points, or point clouds, often leading to substantial memory requirements and the ability to handle only short sequences, we introduce a novel approach for 3D human action recognition, denoted as SpATr (Spiral Auto-encoder and Transformer Network), specifically designed for fixed-topology mesh sequences. The SpATr model disentangles space and time in the mesh sequences. A lightweight auto-encoder, based on spiral convolutions, is employed to extract spatial geometrical features from each 3D mesh. These convolutions are lightweight and specifically designed for fix-topology mesh data. Subsequently, a temporal transformer, based on self-attention, captures the temporal context within the feature sequence. The self-attention mechanism enables long-range dependencies capturing and parallel processing, enabling scalability for long sequences. The proposed method is evaluated on three prominent 3D human action datasets: Babel, MoVi, and BMLrub, from the Archive of Motion Capture As Surface Shapes (AMASS). Our results analysis demonstrates the competitive performance of our SpATr model in 3D human action recognition while maintaining efficient memory usage. The code and the training results are publicly available at https://github.com/h-bouzid/spatr.

Real-time visual SLAM based YOLO-Fastest for dynamic scenes

Within the realm of autonomous robotic navigation, simultaneous localization and mapping (SLAM) serves as a critical perception technology, drawing heightened attention in contemporary research. The traditional SLAM systems perform well in static environments, but in the real physical world, dynamic objects can destroy the static geometric constraints of the SLAM system, further limiting its practical application in the real world. In this paper, a robust dynamic RGB-D SLAM system is proposed to expand the number of static points in the scene by combining with YOLO-Fastest to ensure the effectiveness of the geometric constraints model construction, and then based on that, a new thresholding model is designed to differentiate the dynamic features in the objection bounding box, which takes advantage of the double polyline constraints and the residuals after reprojection to filter the dynamic feature points. In addition, two Gaussian models are constructed to segment the moving objects in the bounding box in the depth image to achieve the effect similar to the instance segmentation under the premise of ensuring the computational speed. In this paper, experiments are conducted on dynamic sequences provided by the TUM dataset to evaluate the performance of the proposed method, and the results show that the root mean squared error metric of the absolute trajectory error of the algorithm of this paper has at least 80% improvement compared to ORB-SLAM2. Higher robustness in dynamic environments with both high and low dynamic sequences compared to DS-SLAM and Dynaslam, and can effectively provide intelligent localization and navigation for mobile robots.

Impact of Brightness on Choroidal Vascularity Index.

The use of choroidal vascularization to diagnose and follow-up ocular and systemic pathologies has been consolidated in recent research. Unfortunately, the choroidal parameters can be different depending on the lighting settings of optical coherence tomography (OCT) images. The purpose of this study was to examine whether the brightness of OCT images could influence the measurements of choroidal parameters obtained by processing and analyzing scientific images with the ImageJ program. In this observational, prospective, non-randomized study, 148 eyes of 74 patients with a mean age of 30.7 ± 8.5 years (ranging from 23 to 61 years) were assessed. All patients underwent a complete ophthalmological examination including slit lamp, fundus oculi, ocular biometry, corneal tomography and spectral domain (SD) OCT evaluations of the foveal region in the enhanced depth imaging (EDI) mode. OCT images at two different brightness levels were obtained. The total choroidal area (TCA), choroidal vascularity index (CVI), stromal choroidal area (SCA) and luminal choroidal area (LCA) at both lower and higher brightness levels were measured. To avoid the bias of operator-dependent error, the lower and higher brightness TCAs were obtained using two methods: the manual tracking mode and fixed area. At the two different brightness levels, LCA, SCA and CVI measurements showed statistically significant changes (p < 0.05), whereas the TCA differences were not statistically significant (p > 0.05). According to the results of this study, highlighting that brightness could affect LCA, SCA and CVI parameters, care should be taken during OCT image acquisition.

Multi-modal deep learning networks for RGB-D pavement waste detection and recognition

To create a clean living environment, governments around the world have hired a large number of workers to clean up waste on pavements, which is inefficient for waste management. To better alleviate this problem, relevant scholars have proposed several deep learning methods based on RGB images to achieve waste detection and recognition. Considering the limitations of color images, we propose an efficient multi-modal learning solution for pavement waste detection and recognition. Specifically, we construct a high-quality outdoor pavement waste dataset called OPWaste, which is more in line with real needs. Compared to other waste datasets, OPWaste dataset not only has the advantages of rich background and high diversity, but also provides color and depth images. Meanwhile, we explore six different multi-modal fusion methods and propose a novel multi-modal multi-scale network (MM-Net) for RGB-D waste detection and recognition. MM-Net introduces a novel multi-scale refinement module (MRM) and multi-scale interaction module (MIM). MRM can effectively refine critical features using attention mechanisms. MIM can gradually realize information interaction between hierarchical features. In addition, we select several representative methods and perform comparative experiments. Experimental results show that MM-Net based on the image addition fusion method outperforms other deep learning models and reaches 97.3% and 84.4% on mAP0.5 and AR metrics. In fact, multi-modal learning plays an important role in intelligent waste recycling. As a promising auxiliary tool, our solution can be applied to intelligent cleaning robots for automatic outdoor waste management.

Multi-scale progressive fusion-based depth image completion and enhancement for industrial collaborative robot applications

Multi-scale progressive fusion-based depth image completion and enhancement for industrial collaborative robot applications

A fractional integral method inverse distance weight-based for denoising depth images

Denoising algorithms for obtaining the effective data of depth images affected by random noise mainly focus on the processing of gray images. These algorithms are not distinct from traditional image-processing methods, and there is no way to evaluate the effectiveness of denoising after the point cloud transformation of denoised depth images. In this paper, the principle of fractional-order integral denoising is studied in detail and inverse distance weighted interpolation is introduced into a denoising model, which is based on the G–L (Grünwald–Letnikov) fractional-order integral to construct a fractional-order integral with an inverse distance weighted denoising model. The model is used to solve the blurring problem caused by sharp changes at the edge and achieves an excellent denoising effect. By using the optimized fractional-order integral denoising operator to construct a denoising model for depth images, the results of the experiments demonstrate that the fractional-order integral of the best denoising effect achieved by the model is −0.6 ≤ ν ≤ −0.4, and the peak signal-to-noise ratio is improved from +6 to +13 dB. In the same condition, median denoising has a distortion of −30 to −15 dB. The depth image that has been denoised is converted into an image of point clouds, and subjective evaluation indicates that the noise is effectively removed. On the whole, the results demonstrate that the fractional-order integral denoising operator with inverse distance weight shows the high efficiency and the outstanding effect in removing noise from depth images while maintaining the image related to the edge and texture information.

Multiscale Cross-Connected Dehazing Network With Scene Depth Fusion.

In this article, we propose a multiscale cross-connected dehazing network with scene depth fusion. We focus on the correlation between a hazy image and the corresponding depth image. The model encodes and decodes the hazy image and the depth image separately and includes cross connections at the decoding end to directly generate a clean image in an end-to-end manner. Specifically, we first construct an input pyramid to obtain the receptive fields of the depth image and the hazy image at multiple levels. Then, we add the features of the corresponding dimensions in the input pyramid to the encoder. Finally, the two paths of the decoder are cross-connected. In addition, the proposed model uses wavelet pooling and residual channel attention modules (RCAMs) as components. A series of ablation experiments shows that the wavelet pooling and RCAMs effectively improve the performance of the model. We conducted extensive experiments on multiple dehazing datasets, and the results show that the model is superior to other advanced methods in terms of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and subjective visual effects. The source code and supplementary are available at https://github.com/CCECfgd/MSCDN-master.

Co-manipulation of soft-materials estimating deformation from depth images

Co-manipulation of soft-materials estimating deformation from depth images

Artifact Removal in Long-Range Single Photon LiDAR via Waveforms-Guided Depth Image Estimation

Artifact Removal in Long-Range Single Photon LiDAR via Waveforms-Guided Depth Image Estimation

Evaluation of retinal and choroidal vascular structures in patients with anxiety disorder.

To evaluate the retinal and choroidal vascular structures in patients with anxiety disorders. Thirthy-four eyes of 34 patients who were diagnosed with any anxiety disorders were compared with 32 eyes of 32 age- and sex-matched controls. Central macular thickness (CMT), foveal vascular zone (FAZ) area, total retinal vascular densities of superficial and deep capillary plexus (VDSCP, VDDCP), outer retinal and choriocapillary layers (ORL, CCL) blood flow rates, central subfoveal choroidal thickness (SFCT) and choriodal vascularity index (CVI) were evaluated with optical coherence tomography angiography (OCT-A) and enhanced depth imaging optical coherence tomography (EDI-OCT). No statistical differences were found between the study and control groups in terms of CMT, FAZ area, VDSCP, VDDCP, ORL and CCL blood flow rates. The mean SFCT was 346.26 ± 64.26 µm in patients with anxiety disorder and was found to be statistically significantly thicker than the control group (319.56 ± 37.19 µm) (p = 0.042). Besides, CVI was significantly lower in the study group (71.09 ± 2.64 vs 73.13 ± 3.31, p = 0.008). In people with anxiety disorders, the SFCT was found to be thicker and CVI was found to be lower than normal subjects. Although anxiety and stress are important factors in central serous chorioretinopathy, multifactorial factors, including ocular factors, play a role in the pathophysiology of the disease. There is a need for prospective studies with larger series on the subject.

Effects of age and axial length on choroidal stratified structure in normal eyes

To quantify the choroidal structures of normal eyes by optical coherence tomography (OCT)-based binarization and evaluate the relationships among age, refractive power, and ocular axial length. This was a retrospective observational study. One hundred and eighty nine eyes of 189 subjects without ocular diseases were examined by enhanced depth imaging (EDI)-OCT. A choroidal OCT horizontal image with a width of 1500 μm centered on the fovea was binarized. The lumen, stroma, and total choroidal area in the choriocapillaris (CC), Sattler’s layer (SL), and Haller's layer (HL) were measured, and the ratio of the luminal area to total choroidal area (L/C ratio) was calculated. Multiple regression analysis was performed for choroidal parameters in each choroidal layer and for age, refractive power, and ocular axial length. Multiple regression analysis showed that an older age was significantly correlated with a lower choroidal area and the L/C ratio in all choroidal layers (each P < 0.05). A Long axial length was significantly associated with lower SL and HL (P < 0.05), but not with refractive power. In the choroid of normal eyes, age-related decreases in the choroidal area and L/C ratio were associated with all choroidal layers, and elongation of the axial length was associated with thinning of SL and HL.

Choroidal Thickness Increases During Parasympathetic Dominance After Immersion of the Foot in Warm Water.

This study aimed to assess the course of changes in choroidal morphology after immersion of the foot in warm water at 40°C using enhanced depth imaging optical coherence tomography (EDI-OCT). Forty-three right eyes of 43 healthy participants were included. Changes in choroidal morphology were determined using EDI-OCT to evaluate subfoveal choroidal thickness (SCT). Systolic, diastolic, and mean blood pressures (SBP, DBP, and MBP, respectively) were also measured to determine systemic circulatory dynamics at baseline, immediately after immersion (0 min), and 10, 20, and 30 min after immersion. Immediately after immersion, SBP, DBP, and MBP were significantly declined versus baseline. In contrast, the SCT was significantly increased after warm water immersion. However, all these parameters did not change significantly compared to the baseline within 30 min. In the normal eye, parasympathetic nerve activity induced by warming stimuli increases choroidal morphology in response to a decrease in systemic circulatory activity, which normalizes within 30 min. The findings of this study may provide basic data for the prevention and treatment of various choroidal diseases in which sympathetic hyperactivity is involved in the pathogenesis.

Evaluation of Macular Neurovasculature and Choroidal Blood Flow Following Inferior Oblique Myectomy Using Optical Coherence Tomography Angiography.

To evaluate the short-term effects of inferior oblique myectomy on the retinal neurovasculature, choroidal thickness, and choroidal vascularity index at the macula. Patients older than 5 years who were candidates for inferior oblique muscle myectomy surgery participated in the study. Patients with any systemic or ocular disease that could affect the macular neurovasculature were not included in the study. After recording demographic data, including age and gender, and conducting a complete ophthalmic examination, macular optical coherence tomography (OCT), enhanced depth imaging OCT, and OCT angiography imaging (AngioVue software (V.2017.1.0.151; Optovue, Inc) were performed before (1 day to 1 week) and in the specific time intervals (1 week, 1 month, and 3 months) after the surgery for all participants. Eighteen patients (13 male and 5 female) who underwent inferior oblique muscle myectomy, with a mean ± standard deviation age of 24.22 ± 18.14 years, were included in this study. The baseline mean ± standard deviation of subfoveal choroidal luminal area and subfoveal total choroidal area were 0.390 ± 0.03 and 0.539 ± 0.04 mm2, respectively. The changing pattern of the subfoveal choroidal vascularity index and subfoveal choroidal luminal area was statistically significant (P = .013 and .035, respectively). Inferior oblique myectomy can lead to changes in choroid hemodynamics in the short term. However, these changes seem to be temporary. [J Pediatr Ophthalmol Strabismus. 2024;61(4):235-244.].

Enhanced robot obstacle avoidance strategy: efficient distance estimation and collision avoidance for hidden robots

Human-robot interaction is crucial for the future of smart factories and new industrial systems. Safety in robotics has always been a top priority, with external sensors being studied to construct safety perception systems for robots. This paper proposes an obstacle avoidance strategy based on an efficient distance estimation method using a vision sensor to address the challenge of robot occlusion. The method fuses depth images with a predefined robot skeleton model to estimate robot pose in real time, and uses the optimized potential field model to achieve full-body collision avoidance. Comparative experiments validate the efficiency of the proposed method, which represents a significant contribution to enhancing human–robot interaction and safety in industrial settings.

Advantages and Pitfalls of the Use of Optical Coherence Tomography for Papilledema.

Papilledema refers to optic disc swelling caused by raised intracranial pressure. This syndrome arises from numerous potential causes, which may pose varying degrees of threat to patients. Manifestations of papilledema range from mild to severe, and early diagnosis is important to prevent vision loss and other deleterious outcomes.The purpose of this review is to highlight the role of optical coherence tomography (OCT) in the diagnosis and management of syndromes of raised intracranial pressure associated with papilledema. Ophthalmoscopy is an unreliable skill for many clinicians. Optical coherence tomography is a non-invasive ocular imaging technique which may fill a current care gap, by facilitating detection of papilledema for those who cannot perform a detailed fundus examination.Optical coherence tomography may help confirm the presence of papilledema, by detecting subclinical peripapillary retinal nerve fiber layer (pRNFL) thickening that might otherwise be missed with ophthalmoscopy. Enhanced depth imaging (EDI) and swept source OCT techniques may identify optic disc drusen as cause of pseudo-papilledema. Macular ganglion cell inner plexiform layer (mGCIPL) values may provide early signs of neuroaxonal injury in patients with papilledema and inform management for patients with syndromes of raised intracranial pressure. There are well-established advantages and disadvantages of OCT that need to be fully understood to best utilize this method for the detection of papilledema.Overall, OCT may complement other existing tools by facilitating detection of papilledema and tracking response to therapies. Moving forward, OCT findings may be included in deep learning models to diagnose papilledema.

View synthesis with multiplane images from computationally generated RGB-D light fields

Image based view synthesis using deep neural networks provide novel scene views from a set of captured single or multiple images. Multiplane images (MPI) represent scene content as set of RGBα planes within a reference view frustum and render novel views by projecting the content into the target viewpoints. Image based view synthesis with multiple images is very popularly deployed in various areas because it effectively represents geometric uncertainty in ambiguous regions and can convincingly simulate non-Lambertian effects. However, previous image based view synthesis approaches suffer from interpolating and extrapolating information in pixels or ray spaces to generate seamless novel views without occlusion. To effectively improve visual performance for view interpolation and extrapolation, this paper proposes a novel view synthesis with MPI images. From a monocular RGB image, light field images are computationally generated, the proposed depth map guided deep network produces robust MPI using the light field images and their corresponding depth images, and the MPI network embedded with depth attention blocks forces semantic and geometric information to be uniformly distributed and divided among layers. The proposed approach achieves 3.5% and 4.02% improvements in SSIM and PSNR values, comparing to the SOTA approaches. Qualitative analysis on benchmark dataset also verifies the robustness of the proposed approach.

Simultaneous evaluation of iris area and subfoveal choroidal thickness in Fuchs uveitis syndrome

Background/aimsTo simultaneously evaluate iris area (IA) and subfoveal choroidal thickness (SFCT) in eyes with Fuchs Uveitis Syndrome (FUS).MethodsWe prospectively recruited a case series of patients with FUS at our institution, simultaneously measuring IA with anterior segment spectral domain optical coherence tomography (SD-OCT) and SFCT with enhanced depth imaging optical coherence tomography (EDI-OCT). Iris images were analyzed by ImageJ software. We tested the differences in intereye IA and SFCT with the healthy eye (HE) using the Wilcoxon test, and clinical interpretation was controlled by intraclass correlation coefficient (ICC) between two masked specialists.ResultsSixteen patients with unilateral FUS were included. Six were female, and the age range was 37 to 67 (median age 48 years, IQR 41–60). ICC of 98.9%, with a lower confidence interval of 97%. Eyes with FUS had a significant thinning of the total iris median area (p < 0.002), restricted to the temporal and nasal areas compared to the HE (p < 0.01 and < 0.001, respectively). SFCT was also significantly thinner compared to the HE (p < 0.0001). A low correlation was found between iris and choroidal thinning in FUS eyes (rs = 0.21; p = 0.4).ConclusionsThis study found reduced iris area and subfoveal choroidal thickness in eyes with FUS compared to the normal fellow eye.

Depth Image Completion through Iterative Low-Pass Filtering

This study introduces a spatial-modulated approach designed to recover missing data in in-depth images. Typically, commercial-grade RGB-D cameras utilize structured light or time-of-flight techniques for capturing scene depth. However, these conventional methods encounter difficulties in acquiring depth data from glossy, transparent, or low-reflective surfaces. Additionally, they are prone to interference from broad-spectrum light sources, resulting in defective areas in the captured data. The generation of dense data is further compromised by the influence of noise. In response to these challenges, we implemented an iterative low-pass filter in the frequency domain, effectively mitigating noise and restoring high-quality depth data across all surfaces. To assess the efficacy of our method, deliberate introduction of significant noise and induced defects in the generated depth images was performed. The experimental results unequivocally demonstrate the promising accuracy, precision, and noise-resilient capabilities of our approach. Our implementation is publicly available on the project’s webpage.