Image Error Research Articles

Method of depth simulation imaging and depth image super-resolution reconstruction for a 2D/3D compatible CMOS image sensor.

This paper presents a depth simulation imaging and depth image super-resolution (SR) method for two-dimensional/three-dimensional compatible CMOS image sensors. A depth perception model is established to analyze the effects of depth imaging parameters and evaluate the real imaging effects. We verify its validity by analyzing the depth error, imaging simulation, and auxiliary physical verification. By means of the depth simulation images, we then propose a depth SR reconstruction algorithm to recover the low-resolution depth maps to the high-resolution depth maps in two types of datasets. With the best situation in depth accuracy kept, the root mean square error (RMSE) of Middlebury dataset images are 0.0156, 0.0179, and 0.0183m. The RMSE of RGB-D dataset images are 0.0223 and 0.0229m. Compared with other listed conventional algorithms, our algorithm reduces the RMSE by more than 16.35%, 17.19%, and 23.90% in the Middlebury dataset images. Besides, our algorithm reduces the RMSE by more than 9.71% and 8.76% in the RGB-D dataset images. The recovery effects achieve optimized results.

Metasurfaces and metalenses

AbstractLenses are widely used as a basic optical element in daily life. They are used in cameras, eyeglasses, microscopes, and many other devices. The lenses are developed based on classical refractive optics, which leads to unavoidable imaging errors such as chromatic as well as spherical aberration and coma. To minimize these, conventional imaging systems usually use multiple lenses of different powers and materials.

Spatial-Temporal Changes of Limoto Wetland Land Use/Cover Before, During and After Restoration Activities in Eastern Uganda

Purpose: Limoto Wetland is an arm of the Mpologoma wetland system in the Kyoga basin and is a vital ecosystem that supports rural livelihoods through the provision of various ecological goods and services. However, this ecosystem has been undergoing rapid degradation arising from competing land uses particularly paddy rice growing. In turn, this led to reduced capacity of the wetland to provide ecosystem services, thereby reducing the resilience of both the ecosystem and the livelihoods of adjacent communities. Interventions through a restoration program that introduced alternative livelihood options were instituted. It’s important to document these changes to obtain insights that can aid decision-making for effective restoration and conservation. This study, therefore, sought to examine the spatial-temporal changes in Limoto wetland land use/cover transitions for the years 2015, 2020, and 2022.
 Methodology: To quantify the wetland changes, remotely sensed imageries for 2015, 2020, and 2022 were utilized in classifying land use and land cover dynamics on Limoto Wetland through the Maximum Likelihood algorithm. A total of 500 points were collected from different land use/cover types and used as reference points to develop the image error matrices. Overall accuracy, producer’s and user’s accuracies as well as Kappa statistics were generated from the error matrices. A Kappa test was carried out to measure the extent of classification accuracy; the Kappa coefficient, K, being a coefficient of agreement. It reflects the difference between the actual agreement of classification with reference data and the agreement expected by chance.
 Findings: Results generally showed that five years after the restoration program (2020), Limoto wetlands regained over 50% of previously converted wetlands to farming. Papyrus coverage also more than doubled from 3.4% to 11.35% of the total wetland coverage. Farms degraded over the past years (1986-2019). However, in 2022, wetland coverage declined drastically as farming took up more than half of the coverage (55%) land, and built-up areas also increased. These changes between 2015-2020 were majorly driven by the introduction of alternative livelihood options which vacated communities from the wetland.
 Recommendation: The changes after 2020 were due to the unsustainability of the livelihood options and the effects of the Covid-19 lockdown which rendered only farming as the socio-economic activity, coupled with movement restrictions that curtailed monitoring and control of activities in wetlands by enforcers. The study recommends community-tailored, tenable, and sustainable alternative livelihood options to influence community-led wetland restoration and conservation to curb the continuous loss of this wetland.

Application of an effective marker-less augmented reality image guidance method in dental implant surgery.

The existing augmented reality (AR) dental implant surgery navigation system usually uses markers to complete image guidance. However, markers often affect dentists' operations and make patients uncomfortable. To solve the problems caused by markers, this paper proposes an effective marker-less image guidance method. After initialisation is completed by contour matching, the corresponding relationship is obtained by matching the feature points between the current frame and the preloaded initial frame. The camera pose is estimated by solving the Perspective-n-Point problem. The registration error of AR images is 0.731±0.144mm. The planting errors are 1.174±0.241mm at the neck, 1.433±0.389mm at the apex and 5.566±2.102° for the angle. The maximum error and standard deviation meet the clinical requirements. We demonstrate that the proposed method can accurately guide dentists to perform dental implant surgery.

A CNN-Based Layer-Adaptive GCPs Extraction Method for TIR Remote Sensing Images

Ground Control Points (GCPs) are of great significance for applications involving the registration and fusion of heterologous remote sensing images (RSIs). However, utilizing low-level information rather than deep features, traditional methods based on intensity and local image features turn out to be unsuitable for heterologous RSIs because of the large nonlinear radiation difference (NRD), inconsistent resolutions, and geometric distortions. Additionally, the limitations of current heterologous datasets and existing deep-learning-based methods make it difficult to obtain enough precision GCPs from different kinds of heterologous RSIs, especially for thermal infrared (TIR) images that present low spatial resolution and poor contrast. In this paper, to address the problems above, we propose a convolutional neural network-based (CNN-based) layer-adaptive GCPs extraction method for TIR RSIs. Particularly, the constructed feature extraction network is comprised of basic and layer-adaptive modules. The former is used to achieve the coarse extraction, and the latter is designed to obtain high-accuracy GCPs by adaptively updating the layers in the module to capture the fine communal homogenous features of the heterologous RSIs until the GCP precision meets the preset threshold. Experimental results evaluated on TIR images of SDGSAT-1 TIS and near infrared (NIR), short wave infrared (SWIR), and panchromatic (PAN) images of Landsat-8 OLI show that the matching root-mean-square error (RMSE) of TIS images with SWIR and NIR images could reach 0.8 pixels and an even much higher accuracy of 0.1 pixels could be reached between TIS and PAN images, which performs better than those of the traditional methods, such as SIFT, RIFT, and the CNN-based method like D2-Net.

An Image Planar Positioning Method Base on Fusion of Dual-View Airborne SAR Data

Effective utilization of airborne synthetic-aperture (Airborne SAR) imagery often requires precise location of each image pixel. Historically, the positioning of airborne SAR imagery either relies on the use of reliable reference points to determine the relative position of the image, or requires the precise motion information of the aircraft and the characteristics of the SAR data collection system as input to determine the absolute position of the image. However, for many applications, the accuracy of traditional positioning methods is not high due to the challenge in obtaining the accurate geographic positions of reliable reference points and the inaccuracy of the recorded aircraft motion information. This study introduces an airborne SAR image planar positioning approach based on the premise that the systematic positioning error of the dual-view airborne SAR images are relatively consistent. The suggested planar positioning method applies the positioning auxiliary parameters of the initial ground-range airborne SAR image to ascertain the transformation relationship between the target’s initial geographic position and pixel position, and it then uses the equivalent equation for the position of the homologue point to assess the systematic positioning error of the SAR image and determine the geographic position of a pixel in a digital SAR image. This approach has advantages over previous techniques in that it requires no precise geographic position information of the ground reference points, and on the basis of using the RD model to accomplish coarse positioning of four corners of SAR image, it no longer needs aircraft trajectory data. Tests were conducted using two airborne SAR images actually captured, and the experimental results indicate that the proposed method can achieve high precision planar positioning of dual-view airborne SAR images. Error sources are analyzed and recommendations are given to improve image positioning accuracy in future airborne SARs.

AAEE‐Net: Attention‐guided aggregation and error‐aware enhancement network for accurate and efficient stereo matching

AbstractStereo matching is a fundamental and long‐standing task in computer vision. Although learning‐based stereo matching algorithms have made remarkable progress, two major challenges still persist. Firstly, existing cost aggregation methods that use stacked three‐dimensional convolutions are complex, leading to heavy computation and memory costs. Secondly these methods continue to struggle with establishing reliable matches in weakly matchable such as that edges and thin structures. To overcome these limitations, we propose an accurate and efficient network called Attention‐guided Aggregation and Error‐aware Enhancement Network (AAEE‐Net). Our approach involves designing an Attention‐guided Aggregation Mechanism (AAM) based on simple image features. This mechanism uses attention weights generated from image features to guide cost aggregation with a more efficient and effective strategy. Additionally, we propose an Error‐aware Enhancement Module (EEM) that refines the raw disparity by combining high‐frequency information from the original image and warp error between the left and right views. EEM enables the network to learn error correction capabilities that produce excellent subtle details and sharp edges. The experimental results on the SceneFlow and KITTI benchmark datasets demonstrate that AAEE‐Net achieves state‐of‐the‐art performance with low inference time. The qualitative results show that AAEE‐Net significantly improves predictions, especially for thin structures.

Effects and elimination of image grating defocusing on a double-Ronchi shearing interference field.

Double-Ronchi shearing interferometry is a promising wavefront aberration measurement system for advanced lithography projection lens systems. The image grating defocusing is a key systematic error of the interferometer. However, the effects and elimination of this error have not been systematically researched. In this work, the interference field effects caused by the image grating defocusing are analyzed based on the theories of scalar diffraction, and a method to eliminate the effects is proposed. The theoretical analysis has been verified by a simulation and experiments. The results show that the error of image grating defocusing is mainly expressed as the Zernike Z4 term and Z9 term in the reconstructed wavefront, and the coefficients of Z4, and Z9, respectively, are related to NA2, NA4, and the defocus distance z. When the numerical aperture (NA) of the under-test projection lens is 0.6, 99.8384% of the errors caused by the image grating defocusing can be removed. When the NA is reduced to 0.3, 99.9854% of the errors can be removed. Additionally, when the NA is less than 0.1, almost all the errors can be eliminated.

PO-2273 Comparison of immobilisation, imaging and set-up errors during head and neck radiotherapy

PO-2273 Comparison of immobilisation, imaging and set-up errors during head and neck radiotherapy

Wheeled Robot Visual Odometer Based on Two-dimensional Iterative Closest Point Algorithm

According to the two-dimensional motion characteristics of planar motion wheeled robot, the visual odometer was dimensionally reduced in this study. In the feature point matching part of visual odometer, the contour constraint was used to filter out the mismatched feature point pairs (abbreviated as FPP). This method could also filter out the matched FPP, and the feature of FPP was correct color image matches, however, their depth image error was large. This offered higher quality matched FPP for the subsequent interframe motion estimation. Dimension reduction was performed in the interframe motion estimation part, and the two-dimensional Iterative Closest Point (ICP) algorithm was used for camera motion estimation. The experiments indicated that the proposed algorithm effectively improved the computational speed and precision of planar motion wheeled robot visual odometer. This research indicates that the dimension reduction processing of ICP algorithm can effectively improve the operation speed and calculation accuracy of planar motion wheeled robot visual odometry, which provides a good reference and data support for the subsequent research of wheeled robot visual odometry in the future.

Analysis of Land Surface Temperature Distribution in Response to Land Use Land Cover Change in Agroforestry Dominated Area, Gedeo Zone, Southern Ethiopia

This study examined LST distribution in Ethiopia's agroforestry-dominated Gedeo Zone due to Land Use Land Cover change. For 2005, 2011, 2017, and 2022, 10 m Sentinel 2A and 30 m Landsat images were used to extract and map LST and LULC distribution. The DOS1 method corrected atmospheric errors in all satellite images. LULC change was detected using SVM image classification. The study result revealed that the Agroforestry and Built-up coverage has increased by 1520 sq. km and 2600 sq. km, respectively, from 2005 to 2022. The Bare Land and Farm Land coverage decreased by 1554 sq. km and 2565 sq. km, respectively, in the same period. The LST result has shown that there has been a remarkable variation in the spatial pattern of the LST between 2005 and 2022. The average LST in Agroforestry, Bare Land, Farm Land, and Built-up area has progressively increased over the years, from 19.6°C, 26.0°C, 20.2°C, and 25.58°C in 2005 to 25°C, 32.16°C, 28.23°C, and 30.62 °C in 2022, respectively. While in 2005, the maximum recorded LST did not exceed 37.3°C, by 2022, it had increased by close to 3°C, reaching 40.6°C. The overall result revealed that the average LST in °C has increased from 2005 to 2022. From the result, it was concluded that agroforestry had contributed a lot to LST distribution. LST may not depend on the local LULC change only; other factors like urbanization and global warming could play a significant role in changing LST locally and globally.

An improved probabilistic diagnosis imaging algorithm for quantifying Hole-edge crack growth

Probabilistic damage imaging (PDI) algorithm based on Lamb wave is one of the most promising structural health monitoring technologies for quantifying the hole-edge crack. However, it is still a challenge how to use a limited number of sensors to quantify the hole-edge crack. This paper proposed an improved probabilistic damage imaging algorithm (IPDI) to track the hole-edge crack growth. A simulation has been conducted to deeply study the interaction of cracks and Lamb waves. A damage index calculated by the frequency-domain energy has been found to have a good capacity for monotonically characterizing crack propagation. The further study derives a unifying relationship between damage index and the ellipse size control factor from the crack to transmitter–receiver paths with different lengths and angles. The unifying relation is used to define a new dynamic elliptical control parameter in PDI algorithm to improve the capacity of crack quantification. The improved PDI algorithm is verified by both of simulation and experiments with different exciting frequencies, whose results show that the crack imaging error of the IPDI algorithm is below 1 mm, and the accuracy is much higher than that of the traditional PDI algorithm. The proposed IPDI algorithm can effectively quantify the hole-edge crack growth.

Malignant melanoma dermoscopy image classification method based on multi‐modal medical features

AbstractSkin cancer is one of the deadliest cancers, and it has been widely developed worldwide since the last decade. Malignant melanoma is currently the most deadly skin cancer. If malignant melanoma is diagnosed at an early stage, the probability of patients being cured will be greatly improved. At present, most existing skin lesion image classification methods only use deep learning. However, the multi‐modal features of skin lesions in the medical domain are not well utilized and integrated. To reduce the classification error of the skin lesion images caused by the complexity and subjectivity of visual interpretation, a malignant melanoma dermoscopy image classification method based on multi‐modal medical features is proposed in this paper which is inspired by the fuzzy decision‐making process of doctors. It can reduce the subjective difference in the image classification process and assist dermatologists to analyze the skin lesion area. Firstly, the feature detection method based on the extension theory can effectively quantify the difference between different colour features. Then, an interpretable segmentation edge of the skin lesion is established by using the neutrosophic theory which can convert the image into the neutrosophic space. The edge of the skin lesion is captured by applying the Hierarchical Gaussian Mixture Model (HGMM) method. Next, the edge sequence is established by segmenting the edge, and the contour regularity, symmetry, and uniformity of the edge of the skin lesion are analyzed. Finally, the extracted multi‐feature sets are used for dermoscopy image classification. Experiments are carried out on real datasets, and the classification accuracy of four kernel functions is verified. The experimental results show that the authors’ method can effectively improve the classification accuracy of benign dermoscopy images and malignant dermoscopy images.

Efficacy of CAD/CAM technology in dental procedures performed by students: A systematic scoping review of randomized clinical trials

ObjectivesTo assess the efficacy of computer-aided design/computer-aided manufacturing (CAD/CAM) in dental procedures performed by dental students. Methods: A systematic scoping review was performed. MeSH terms and keywords were specified to assess randomized clinical trials (RCTs) in all idioms. Results: Nine RCTs that include 520 students were selected. Two RCTs studied implant placement, and the rest of the trials explored implant impressions, intraoral scans, canal obturation, head positioning errors in panoramic images, deep caries and pulp exposure, diagnosis of temporomandibular disorders, and a tooth for restorations. Considering efficacy, variable results were found. The RCTs that explored implant impressions, deep caries and pulp exposure, and diagnosis of temporomandibular disorders showed better results when CAD/CAM technology was performed by students. Two RCTs that investigated implant placement presented contradictory outcomes; one showed better results when CAD/CAM systems were used while the other described similar results to conventional methods. The remaining RCTs did not find differences with conventional procedures. Considering the time spent during the procedures, two studies showed no differences between groups, while another indicated that the time spent was less for digital impressions (p < 0.001). A continuous decrease in scanning time was also perceived for all groups as skills in intraoral scanning augmented (p = 0.0005). Students indicated an interest in utilizing the CAD/CAM systems and recognized that their abilities were enhanced. Conclusions: Considering the limitations of this review and the limited number of RCTs evaluated, students recognized that their skills improved using CAD/CAM systems; however, only in some specific procedures greater efficacy and less time spent were observed when this technology was used. Dental students also showed great interest in using these techniques.

On the correction of respiratory motion-induced image reconstruction errors in positron-emission tomography-guided radiation therapy

Free breathing (FB) positron emission tomography (PET) images are routinely used in radiotherapy for lung cancer patients. Respiration-induced artifacts in these images compromise treatment response assessment and obstruct clinical implementation of dose painting and PET-guided radiotherapy. The purpose of this study is to develop a blurry image decomposition (BID) method to correct motion-induced image-reconstruction errors in FB-PETs. Assuming a blurry PET is represented as an average of multi-phase PETs. A four-dimensional computed-tomography image is deformably registered from the end-inhalation (EI) phase to other phases. With the registration-derived deformation maps, PETs at other phases can be deformed from a PET at the EI phase. To reconstruct the EI-PET, the difference between the blurry PET and the average of the deformed EI-PETs is minimized using a maximum-likelihood expectation-maximization algorithm. The developed method was evaluated with computational and physical phantoms as well as PET/CT images acquired from three patients. The BID method increased the signal-to-noise ratio from 1.88±1.05 to 10.5±3.3 and universal-quality index from 0.72±0.11 to 1.0 for the computational phantoms, and reduced the motion-induced error from 69.9% to 10.9% in the maximum of activity concentration and from 317.5% to 8.7% in the full width at half maximum of the physical PET-phantom. The BID-based corrections increased the maximum standardized-uptake values by 17.7±15.4% and reduced tumor volumes by 12.5±10.4% on average for the three patients. The proposed image-decomposition method reduces respiration-induced errors in PET images and holds potential to improve the quality of radiotherapy for thoracic and abdominal cancer patients.

On Development of Reliable Machine Learning Systems Based on Machine Error Tolerance of Input Images

With the rapid development of machine learning technologies, more and more practical applications arise. Representative machine learning techniques that receive much attention include object detection and image classification, which can be applied to many applications, such as self-driving cars, traffic flow calculation, and detection of product defects in factories. In this article, we investigate tolerability of errors for input images in machine learning systems and develop a generic reliability enhancement methodology. This work is based on our preliminary studies on image classification, but we put major focuses in object detection applications and the comprehensive comparisons to the prior studies. The first one error-tolerability test method to support reliability enhancement of object detection applications is then proposed based on careful error-tolerability examination of input images. The experimental results show that the test accuracy of this method can achieve 93.06%, which is the state-of-the-art. One special advantage of the proposed method is that unlike the previous error-tolerance methods in the literature, no golden reference data are required for acceptability determination by the proposed method. Hence, on-line testing can be supported. Our method is also implemented and validated in hardware. The results show that the hardware performance is up to 192 frames per second (FPS), which can thus also support real-time operations.

Image-Based Visual Servoing of Quadrotors to Arbitrary Flight Targets

Visual servoing of Unmanned Aerial Vehicles (UAVs) has achieved satisfactory performance in fixed and planar motion targets. Due to highly coupled system dynamics and the sensitivity of the target image to aircraft attitude, the problem for chasing free-flying targets remains challenging. In this paper, a vision-based algorithm is designed for controlling an UAV while tracking an intruder flying arbitrarily in 3D space. Image-based visual servoing is used to design controllers that depend directly on errors in image plane. Specifically, a virtual camera approach is adopted to decouple the UAV dynamics by compensating the pitch and yaw, and an improved image error term is proposed to reduce the impact of the UAV rotation on error signals in the process of tracking, thus a simplified control design is achieved, and the stability of the visual servo system is guaranteed. Comparison and ablation experiments in both simulated and real environments are provided to verify the effectiveness of the proposed method.

Adaptive event-triggered control for uncalibrated visual servoing of robot manipulators

In the study, an adaptive event-triggered control strategy is proposed for image-based visual servoing of eye-to-hand robot manipulators, where the camera used does not need to be calibrated, and the dynamics behavior of manipulator is considered in design and analysis. To address the uncertainty in camera parameters and the nonlinearity in robot dynamics, and accommodate the errors between the real-time control signals and the piecewise-constant event-triggered control signals, a new and novel robust adaptive estimation approach is developed, and well fused with visual feedback control design so that the boundedness of all closed-loop signals, and the asymptotic convergence of image error towards zero are established simultaneously. Besides rigorous proof in theory, the obtained results are also confirmed by comparative simulation tests.

Deep learning for improving PET/CT attenuation correction by elastic registration of anatomical data.

For PET/CT, the CT transmission data are used to correct the PET emission data for attenuation. However, subject motion between the consecutive scans can cause problems for the PET reconstruction. A method to match the CT to the PET would reduce resulting artifacts in the reconstructed images. This work presents a deep learning technique for inter-modality, elastic registration of PET/CT images for improving PET attenuation correction (AC). The feasibility of the technique is demonstrated for two applications: general whole-body (WB) imaging and cardiac myocardial perfusion imaging (MPI), with a specific focus on respiratory and gross voluntary motion. A convolutional neural network (CNN) was developed and trained for the registration task, comprising two distinct modules: a feature extractor and a displacement vector field (DVF) regressor. It took as input a non-attenuation-corrected PET/CT image pair and returned the relative DVF between them-it was trained in a supervised fashion using simulated inter-image motion. The 3D motion fields produced by the network were used to resample the CT image volumes, elastically warping them to spatially match the corresponding PET distributions. Performance of the algorithm was evaluated in different independent sets of WB clinical subject data: for recovering deliberate misregistrations imposed in motion-free PET/CT pairs and for improving reconstruction artifacts in cases with actual subject motion. The efficacy of this technique is also demonstrated for improving PET AC in cardiac MPI applications. A single registration network was found to be capable of handling a variety of PET tracers. It demonstrated state-of-the-art performance in the PET/CT registration task and was able to significantly reduce the effects of simulated motion imposed in motion-free, clinical data. Registering the CT to the PET distribution was also found to reduce various types of AC artifacts in the reconstructed PET images of subjects with actual motion. In particular, liver uniformity was improved in the subjects with significant observable respiratory motion. For MPI, the proposed approach yielded advantages for correcting artifacts in myocardial activity quantification and potentially for reducing the rate of the associated diagnostic errors. This study demonstrated the feasibility of using deep learning for registering the anatomical image to improve AC in clinical PET/CT reconstruction. Most notably, this improved common respiratory artifacts occurring near the lung/liver border, misalignment artifacts due to gross voluntary motion, and quantification errors in cardiac PET imaging.

Quantification of the methodological error in kinematic evaluation of the DRUJ using dynamic CT

Distal radio-ulnar joint (DRUJ) motion analysis using dynamic CT is gaining popularity. Following scanning and segmentation, 3D bone models are registered to (4D-)CT target frames. Imaging errors like low signal-to-noise ratio (SNR), limited Z-coverage and motion artefacts influence registration, causing misinterpretation of joint motion. This necessitates quantification of the methodological error. A cadaver arm and dynamic phantom were subjected to multiple 4D-CT scans, while varying tube charge-time product and phantom angular velocity, to evaluate the effects of SNR and motion artefacts on registration accuracy and precision. 4D-CT Z-coverage is limited by the scanner. To quantify the effects of different Z-coverages on registration accuracy and precision, 4D-CT was simulated by acquiring multiple spiral 3D-CT scans of the cadaver arm. Z-coverage was varied by clipping the 3D bone models prior to registration. The radius position relative to the ulna was obtained from the segmentation image. Apparent relative displacement seen in the target images is caused by registration errors. Worst-case translations were 0.45, 0.08 and 1.1 mm for SNR-, Z-coverage- and motion-related errors respectively. Worst-case rotations were 0.41, 0.13 and 6.0 degrees. This study showed that quantification of the methodological error enables composition of accurate and precise DRUJ motion scanning protocols.