Image Pairs Research Articles

Multi-scale transformer network for super-resolution of visible and thermal air images

Reference image-based Super-Resolution (RefSR) is introduced to improve the quality of a Low-resolution (LR) input image by leveraging the additional information provided by a High-Resolution (HR) reference image (Ref). While existing RefSR methods focus on thermal or visible flows separately, they often struggle to enhance the resolution of small objects such as Mini/Micro UAVs (Unmanned Aerial Vehicle) due to the resolution disparities between the input and reference images. To cope with these challenges when dealing with UAV early detection in context of video surveillance, we propose ThermoVisSR, a multiscale texture transformer for enhancing the Super-Resolution (SR) of visible and thermal images of Mini/Micro UAVs. Our approach tries to reconstruct the fine details of these objects while preserving their approximation (the body form and color of the different scene objects) already contained in the LR image. Hence, our model is divided up into two streams dealing separately with approximation and detail reconstruction. In the first one, we introduce a Convolution Neural Network (CNN) fusion backbone to extract the Low-Frequency (LF) approximation from the original LR image pairs. In the second one and to extract the details from the Ref image, our approach involves blending features from both visible and thermal sources to make the most of what each offer. Subsequently, we introduce the High-Frequency Texture Transformer (HFTT) across various resolutions of the merged features to ensure an accurate correspondence matching and significant transfer of High-Frequency (HF) patches from Ref to LR images. Moreover, to adapt the injection to the different bands well, we incorporate the separable software decoder (SSD) into the HFTT allowing to capture channel-specific details during the reconstruction phase. We validated our approach using a newly created dataset of Air images of Mini/Micro UAVs. Experimental results demonstrate that the proposed model consistently outperforms the state-of-the-art approaches on both qualitative and quantitative assessments.

Unsupervised low-light image enhancement by data augmentation and contrastive learning

ABSTRACT Today, with the increasing demand for visual perception and high-level computational vision tasks, the field of low-light enhancement is rapidly developing. However, models trained on existing datasets often fail or suffer significant performance degradation in real-world low-light scenarios. This performance degradation is frequently due to the limitations of current databases, which typically contain small quantities of paired images of a single type. This article proposes an unsupervised model with a unique data augmentation technique that transforms a regular image database into a paired image database. By adjusting image parameters during training to change exposure, a regular image database can be converted into a paired one. The model restores low-exposure images by extracting lighting features through comparative learning. Evaluations of the LOL and DIV2K datasets demonstrate the proposed model's effectiveness, achieving notable results in low-light image enhancement. This method removes dataset restrictions, broadening the model's range of applications.

Longitudinal registration of thoracic CT images with radiation-induced lung diseases: A divide-and-conquer approach based on component structure wise registration using coherent point drift

Background and ObjectiveRegistration of pulmonary computed tomography (CT) images with radiation-induced lung diseases (RILD) was essential to investigate the voxel-wise relationship between the formation of RILD and the radiation dose received by different tissues. Although various approaches had been developed for the registration of lung CTs, their performances remained clinically unsatisfactory for registration of lung CT images with RILD. The main difficulties arose from the longitudinal change in lung parenchyma, including RILD and volumetric change of lung cancers, after radiation therapy, leading to inaccurate registration and artifacts caused by erroneous matching of the RILD tissues. MethodsTo overcome the influence of the parenchymal changes, a divide-and-conquer approach rooted in the coherent point drift (CPD) paradigm was proposed. The proposed method was based on two kernel ideas. One was the idea of component structure wise registration. Specifically, the proposed method relaxed the intrinsic assumption of equal isotropic covariances in CPD by decomposing a lung and its surrounding tissues into component structures and independently registering the component structures pairwise by CPD. The other was the idea of defining a vascular subtree centered at a matched branch point as a component structure. This idea could not only provide a sufficient number of matched feature points within a parenchyma, but avoid being corrupted by the false feature points resided in the RILD tissues due to globally and indiscriminately sampling using mathematical operators. The overall deformation model was built by using the Thin Plate Spline based on all matched points. ResultsThis study recruited 30 pairs of lung CT images with RILD, 15 of which were used for internal validation (leave-one-out cross-validation) and the other 15 for external validation. The experimental results showed that the proposed algorithm achieved a mean and a mean of maximum 1 % of average surface distances <2 and 8 mm, respectively, and a mean and a maximum target registration error <2 mm and 5 mm on both internal and external validation datasets. The paired two-sample t-tests corroborated that the proposed algorithm outperformed a recent method, the Stavropoulou's method, on the external validation dataset (p < 0.05). ConclusionsThe proposed algorithm effectively reduced the influence of parenchymal changes, resulting in a reasonably accurate and artifact-free registration.

Eye-Tracking Methodology to Detect Differences in Attention to Faces Between Developmental Language Disorder and Autism.

Developmental language disorder (DLD) and autism sometimes appear as overlapping conditions in behavioral tests. There is much literature on the visual scanning pattern (VSP) of faces in autistic children, but this is scarce regarding those with DLD. The purpose of this study was to compare the VSP of faces in young children with DLD, those with autism, and typically developing peers, assessing the effect of three variables. Two eye-tracking experiments were designed to assess the effect of the emotion and the poser's gender (Experiment 1) and the poser's age (Experiment 2) on the VSP of participants (Experiment 1: N = 59, age range: 32-74 months; Experiment 2: N = 58, age range: 32-74 months). We operationalized the VSP in terms of attentional orientation, visual preference, and depth of processing of each sort of face. We developed two paired preference tasks in which pairs of images of faces showing different emotions were displayed simultaneously to compete for children's attention. Data analysis revealed two VSP markers common to both disorders: (a) superficial processing of faces and (b) late orientation to angry and child faces. Moreover, one specific marker for each condition was also found: typical preference for child faces in children with DLD versus diminished preference for them in autistic children. Considering the similarities found between children with DLD and those with autism, difficulties of children with DLD in attention to faces have been systematically underestimated. Thus, more effort must be made to identify and respond to the needs of this population. Clinical practice may benefit from the potential of eye-tracking methodology and the analysis of the VSP to assess attention to faces in both conditions. This would also contribute to the improvement of early differential diagnosis in the long run.

O-PRESS: Boosting OCT axial resolution with Prior guidance, Recurrence, and Equivariant Self-Supervision

Optical coherence tomography (OCT) is a noninvasive technology that enables real-time imaging of tissue microanatomies. The axial resolution of OCT is intrinsically constrained by the spectral bandwidth of the employed light source while maintaining a fixed center wavelength for a specific application. Physically extending this bandwidth faces strong limitations and requires a substantial cost. We present a novel computational approach, called as O-PRESS, for boosting the axial resolution of OCT with Prior guidance, a Recurrent mechanism, and Equivariant Self-Supervision. Diverging from conventional deconvolution methods that rely on physical models or data-driven techniques, our method seamlessly integrates OCT modeling and deep learning, enabling us to achieve real-time axial-resolution enhancement exclusively from measurements without a need for paired images. Our approach solves two primary tasks of resolution enhancement and noise reduction with one treatment. Both tasks are executed in a self-supervised manner, with equivariance imaging and free space priors guiding their respective processes. Experimental evaluations, encompassing both quantitative metrics and visual assessments, consistently verify the efficacy and superiority of our approach, which exhibits performance on par with fully supervised methods. Importantly, the robustness of our model is affirmed, showcasing its dual capability to enhance axial resolution while concurrently improving the signal-to-noise ratio.

A lossless probabilistic image encryption algorithm based on QSD decomposition and matrix transformations

With the development of Internet technology, images have become widely used in people's daily life. However, some images carry significant importance and sensitivity, making the security and protection of digital information more crucial. Standard encryption schemes such as RSA, DES, and AES, which have proven effective for textual data, face limitations when applied to images due to their unique features, such as large data capacity and strong pixel correlation. In recent years, numerous effective image encryption schemes utilizing diverse techniques have been proposed. However, the use of integer decomposition in encryption techniques is still underexplored. This paper introduces a novel, lossless, and probabilistic image encryption scheme designed to enhance the security of sensitive images during transmission and storage. This scheme is based on a new integer decomposition technique and utilizes a key with a very large key space. In the proposed approach, an image is encrypted into a pair of images using quasi-square decomposition and new matrix transformations. Security is further reinforced through the use of orthogonal matrices, which enhance diffusion while ensuring data integrity and preservation. Experimental results demonstrate that the proposed scheme is highly secure against most known attacks and cryptanalysis methods. Additionally, a comparison with various existing image encryption techniques shows the effectiveness of the proposed approach. This research provides a significant contribution by introducing new techniques that leverage integer decomposition, which are highly effective against CPA (Chosen Plaintext Attack) and KPA (Known-Plaintext Attack). Moreover, these techniques enhance the entropy value of encrypted images and improve the values of NPCR and UACI, which help prevent differential attacks. The proposed scheme has the potential to advance applications in secure image transmission, storage, and privacy protection.

CorMatcher: A corners-guided graph neural network for local feature matching

Local feature matching is crucial for various computer vision tasks like Structure from Motion (SfM) and visual localization. In human visual matching, individuals typically scan image pairs to detect and match prominent corners, then refine matches by leveraging the geometric structure information from these corners. Inspired by this observation, we propose CorMatcher, a corners-guided detector-free network that investigates the guiding role of prominent corners in local feature matching. The concept of CorMatcher is utilizing the corners-images interaction module (CIIM) and deep supervision technique (DST) to replicate the aforementioned human behaviors, thus generating more human-intuitive features for accurate feature matching. CIIM imitates human behavior that first observes corners to guide the matching process, and then extend the matching space from corners to entire images. Technically, In particular, CIIM first integrates global information across all keypoints to ensure overall perception. Then, CIIM extracts reliable corners, performs feature enhancement among them, and propagates the enhanced features of corners to entire images. Finally, CIIM interacts features of all keypoints to ensure that the global information is comprehensively considered to establish final matches. In addition, we apply DST to the corners, rather than all keypoints, to supervise matching results after each feature enhancement process, thus simulating the human behavior that appraises matching results after each observation. Comprehensive experiments demonstrate that CorMatcher exhibits superior performance compared to several benchmarks.

Dimma: Semi-Supervised Low-Light Image Enhancement with Adaptive Dimming.

Enhancing low-light images with natural colors poses a challenge due to camera processing variations and limited access to ground-truth lighting conditions. To address this, we propose Dimma, a semi-supervised approach that aligns with any camera using a small set of image pairs captured under extreme lighting conditions. Our method employs a convolutional mixture density network to replicate camera-specific noise present in dark images. We enhance results further by introducing a conditional UNet architecture based on user-provided lightness values. Trained on just a few real image pairs, Dimma achieves competitive results compared to fully supervised state-of-the-art methods trained on large datasets.

A vision-language foundation model for the generation of realistic chest X-ray images.

The paucity of high-quality medical imaging datasets could be mitigated by machine learning models that generate compositionally diverse images that faithfully represent medical concepts and pathologies. However, large vision-language models are trained on natural images, and the diversity distribution of the generated images substantially differs from that of medical images. Moreover, medical language involves specific and semantically rich vocabulary. Here we describe a domain-adaptation strategy for large vision-language models that overcomes distributional shifts. Specifically, by leveraging publicly available datasets of chest X-ray images and the corresponding radiology reports, we adapted a latent diffusion model pre-trained on pairs of natural images and text descriptors to generate diverse and visually plausible synthetic chest X-ray images (as confirmed by board-certified radiologists) whose appearance can be controlled with free-form medical text prompts. The domain-adaptation strategy for the text-conditioned synthesis of medical images can be used to augment training datasets and is a viable alternative to the sharing of real medical images for model training and fine-tuning.

Deeper, Sharper, Faster: Application of Efficient Transformer to Galaxy Image Restoration

The Transformer architecture has revolutionized the field of deep learning over the past several years in diverse areas, including natural language processing, code generation, image recognition, and time-series forecasting. We propose to apply Zamir et al.'s efficient transformer to perform deconvolution and denoising to enhance astronomical images. We conducted experiments using pairs of high-quality images and their degraded versions, and our deep learning model demonstrates exceptional restoration of photometric, structural, and morphological information. When compared with the ground-truth James Webb Space Telescope images, the enhanced versions of our Hubble Space Telescope–quality images reduce the scatter of isophotal photometry, Sérsic index, and half-light radius by factors of 4.4, 3.6, and 4.7, respectively, with Pearson correlation coefficients approaching unity. The performance is observed to degrade when input images exhibit correlated noise, point-like sources, and artifacts. We anticipate that this deep learning model will prove valuable for a number of scientific applications, including precision photometry, morphological analysis, and shear calibration.

Frequency-Aware Divide-and-Conquer for Efficient Real Noise Removal.

Deep-learning-based approaches have achieved remarkable progress for complex real scenario denoising, yet their accuracy-efficiency tradeoff is still understudied, particularly critical for mobile devices. As real noise is unevenly distributed relative to underlay signals in different frequency bands, we introduce a frequency-aware divide-and-conquer strategy to develop a frequency-aware denoising network (FADN). FADN is materialized by stacking frequency-aware denoising blocks (FADBs), in which a denoised image is progressively predicted by a series of frequency-aware noise dividing and conquering operations. For noise dividing, FADBs decompose the noisy and clean image pairs into low-and high-frequency representations via a wavelet transform (WT) followed by an invertible network and recover the final denoised image by integrating the denoised information from different frequency bands. For noise conquering, the separated low-frequency representation of the noisy image is kept as clean as possible by the supervision of the clean counterpart, while the high-frequency representation combining the estimated residual from the successive FADB is purified under the corresponding accompanied supervision for residual compensation. Since our FADN progressively and pertinently denoises from frequency bands, the accuracy-efficiency tradeoff can be controlled as a requirement by the number of FADBs. Experimental results on the SIDD, DND, and NAM datasets show that our FADN outperforms the state-of-the-art methods by improving the peak signal-to-noise ratio (PSNR) and decreasing the model parameters. The code is released at https://github.com/NekoDaiSiki/FADN.

A visible-infrared person re-identification method based on meta-graph isomerization aggregation module

Due to different imaging principles of visible-infrared cameras, there are modal differences between similar person images. For visible-infrared person re-identification (VI-ReID), existing works focus on extracting and aligning cross-modal global descriptors in the shared feature space, while ignoring local variations and graph-structural correlations in cross-modal image pairs. In order to bridge the modal differences with graph-structure between key-parts, a Meta-Graph Isomerization Aggregation Module (MIAM) is proposed, which includes Meta-Graph Node Isomerization Module (MNI) and Dual Aggregation Module (DA). In order to completely describe discriminative in-graph local features, MNI establishes meta-secondary cyclic isomorphism relations of in-graph local features by using a multi-branch embedding generation mechanism. Then, the local features not only contain the limited information of the fixed regions, but also benefit from the neighboring regions. Meanwhile, the secondary node generation process considers similar and different nodes of pedestrian graph structure to reduce the interference of identity differences. In addition, the Dual Aggregation (DA) module combines spatial self-attention and channel self-attention to establish the interdependence of modal heterogeneous graph structure pairs and achieve inter-modal feature aggregation. To match heterogeneous graph structures, a Node Center Joint Mining Loss (NCJM Loss) is proposed to constrain the distance between the node centers of heterogeneous graphs. The experiments performed on the SYSU-MM01, RegDB and LLCM public datasets demonstrate that the proposed method performs excellently in VI-ReID.

Roof plane parsing towards LoD-2.2 building reconstruction based on joint learning using remote sensing images

Building models are important for urban studies. Remote sensing multi-spectral (MS) images are widely used for its rich semantic information. The lack of geometry features is fulfilled by introducing photogrammetry derived digital surface models (DSMs), resulting in pairs of DSMs and MS images. Utilizing such pairs and a convolutional neural network, level of detail (LoD) 2.2 building models, which contain roof planes and major roof elements (e.g. dormers), are reconstructed in this work. Leveraging both raster and vector predictions, 3-D building models with straight edges and sharp corners are obtained. The proposed two-stage method first extracts vectorized roof lines from pairs of DSMs and RGB images, followed by generation of detailed 2-D and 3-D polygonal building models. We conducted our experiments based on two datasets: a custom dataset in Landsberg am Lech in Germany, and an open dataset named Roof3D. For the custom dataset, our proposed model achieved mean average precision (mAP) for building roof vertices of 64.3% and for building roof lines of 54.5% at highest. Mean precision and recall for reconstructed 2-D building roof plane polygons are 52.2% and 54.7% respectively. For the Roof3D dataset, mAP is reported to be 25.3% and 12.4% for the extracted building roof lines and roof plane polygons respectively.

A high-quality self-supervised image denoising method based on SDDW-GAN and CHRNet

Image denoising remains a classic and crucial issue in the field of image processing, significantly impacting the outcomes of subsequent image processing tasks. For instance, the denoising network depends on “noise-clean” image pairs to train network effectively. However, it is often hampered by issues such as useful information loss, low training efficiency, and poor blind denoising. To address these challenges, this study proposes a novel image denoising network that integrates the complementary strengths of model-based and learning-based approaches, specifically leveraging the capabilities of both SDDW-GAN and CHRNet. Firstly, SDDW-GAN is designed to estimate the noise distribution on the input noisy images, and a fast-smoothing noisy block sampling algorithm is proposed to extract the noise blocks in noisy images in SDDW-GAN. Secondly, a network with dual generators and dual discriminators based on W-GAN is designed to estimate the noise distribution on the input noisy images and generate noise sample pairs with the same noise distribution, which solves the problem of relying on “noise-clean” image pairs. Thirdly, CHRNet is designed to compute the mapping relationship between the double-noise samples and the single-noise samples. In order to further improve the denoising effect, the dual-channel residual attention module is proposed for fusion learning of global and local features. Experimental results show that the proposed method has a better denoising effect in complex environments and outperforms existing denoising methods. Specifically, in comparison with the stand-alone denoising methods BM3D, DnCNN, Noise2Noise, and Blind2Unblind, the proposed method improves the average peak signal-to-noise ratio (PSNR) by 0.23 dB to 0.78 dB on two benchmarking datasets crossing different noise levels. Its denoising effect is also greater than other competitive stand-alone and combination methods. The proposed method can also extend to low-light image enhancement, deblurring, and super-resolution.

Super-resolution reconstruction for early cervical cancer magnetic resonance imaging based on deep learning

This study aims to develop a super-resolution (SR) algorithm tailored specifically for enhancing the image quality and resolution of early cervical cancer (CC) magnetic resonance imaging (MRI) images. The proposed method is subjected to both qualitative and quantitative analyses, thoroughly investigating its performance across various upscaling factors and assessing its impact on medical image segmentation tasks. The innovative SR algorithm employed for reconstructing early CC MRI images integrates complex architectures and deep convolutional kernels. Training is conducted on matched pairs of input images through a multi-input model. The research findings highlight the significant advantages of the proposed SR method on two distinct datasets at different upscaling factors. Specifically, at a 2× upscaling factor, the sagittal test set outperforms the state-of-the-art methods in the PSNR index evaluation, second only to the hybrid attention transformer, while the axial test set outperforms the state-of-the-art methods in both PSNR and SSIM index evaluation. At a 4× upscaling factor, both the sagittal test set and the axial test set achieve the best results in the evaluation of PNSR and SSIM indicators. This method not only effectively enhances image quality, but also exhibits superior performance in medical segmentation tasks, thereby providing a more reliable foundation for clinical diagnosis and image analysis.

Revolutionizing Bladder Health: Artificial-Intelligence-Powered Automatic Measurement of Bladder Volume Using Two-Dimensional Ultrasound.

Measuring elevated post-void residual volume is important for diagnosing urinary outflow tract obstruction and cauda equina syndrome. Catheter placement is exact but painful, invasive, and may cause infection, whereas an ultrasound is accurate, painless, and safe. The purpose of this single-center study is to evaluate the accuracy of a module for artificial-intelligence (AI)-based fully automated bladder volume (BV) prospective measurement using two-dimensional ultrasound images, as compared with manual measurement by expert sonographers. Pairs of transverse and longitudinal bladder images were obtained from patients evaluated in an urgent care clinic. The scans were prospectively analyzed by the automated module using the prolate ellipsoid method. The same examinations were manually measured by a blinded expert sonographer. The two methods were compared using the Pearson correlation, kappa coefficients, and the Bland-Altman method. A total of 111 pairs of transverse and longitudinal views were included. A very strong correlation was found between the manual BV measurements and the AI-based module with r = 0.97 [95% CI: 0.96-0.98]. The specificity and sensitivity for the diagnosis of an elevated post-void residual volume using a threshold ≥200 mL were 1.00 and 0.82, respectively. An almost-perfect agreement between manual and automated methods was obtained (kappa = 0.85). Perfect reproducibility was found for both inter- and intra-observer agreements. This AI-based module provides an accurate automated measurement of the BV based on ultrasound images. This novel method demonstrates a very strong correlation with the gold standard, making it a potentially valuable decision-support tool for non-experts in acute settings.

Comparing the Output of an Artificial Intelligence Algorithm in Detecting Radiological Signs of Pulmonary Tuberculosis in Digital Chest X-Rays and Their Smartphone-Captured Photos of X-Ray Films: Retrospective Study.

Artificial intelligence (AI) based computer-aided detection devices are recommended for screening and triaging of pulmonary tuberculosis (TB) using digital chest x-ray (CXR) images (soft copies). Most AI algorithms are trained using input data from digital CXR Digital Imaging and Communications in Medicine (DICOM) files. There can be scenarios when only digital CXR films (hard copies) are available for interpretation. A smartphone-captured photo of the digital CXR film may be used for AI to process in such a scenario. There is a gap in the literature investigating if there is a significant difference in the performance of AI algorithms when digital CXR DICOM files are used as input for AI to process as opposed to photos of the digital CXR films being used as input. The primary objective was to compare the agreement of AI in detecting radiological signs of TB when using DICOM files (denoted as CXRd) as input versus when using smartphone-captured photos of digital CXR films (denoted as CXRp) with human readers. Pairs of CXRd and CXRp images were obtained retrospectively from patients screened for TB. AI results were obtained using both the CXRd and CXRp files. The majority consensus on the presence or absence of TB in CXR pairs was obtained from a panel of 3 independent radiologists. The positive and negative percent agreement of AI in detecting radiological signs of TB in CXRd and CXRp were estimated by comparing with the majority consensus. The distribution of AI probability scores was also compared. A total of 1278 CXR pairs were analyzed. The positive percent agreement of AI was found to be 92.22% (95% CI 89.94-94.12) and 90.75% (95% CI 88.32-92.82), respectively, for CXRd and CXRp images (P=.09). The negative percent agreement of AI was 82.08% (95% CI 78.76-85.07) and 79.23% (95% CI 75.75-82.42), respectively, for CXRd and CXRp images (P=.06). The median of the AI probability score was 0.72 (IQR 0.11-0.97) in CXRd and 0.72 (IQR 0.14-0.96) in CXRp images (P=.75). We did not observe any statistically significant differences in the output of AI in digital CXRs and photos of digital CXR films.

DAW-FA: Domain-aware adaptive weighting with fine-grain attention for unsupervised MRI harmonization

Magnetic resonance (MR) imaging often lacks standardized acquisition protocols across various sites, leading to contrast variations that reduce image quality and hinder automated analysis. MR harmonization improves consistency by integrating data from multiple sources, ensuring reproducible analysis. Recent advances leverage image-to-image translation and disentangled representation learning to decompose anatomical and contrast representations, achieving consistent cross-site harmonization. However, these methods face two significant drawbacks: imbalanced contrast availability during training affects adaptation performance, and insufficient utilization of spatial variability in local anatomical structures limits model adaptability to different sites. To address these challenges, we propose Domain-aware Adaptive Weighting with Fine-Grain Attention (DAW-FA) for Unsupervised MRI Harmonization. DAW-FA incorporates an adaptive weighting mechanism and enhanced self-attention to mitigate MR contrast imbalance during training and account for spatial variability in local anatomical structures. This facilitates robust cross-site harmonization without requiring paired inter-site images. We evaluated DAW-FA on MR datasets with varying scanners and acquisition protocols. Experimental results show DAW-FA outperforms existing methods, with an average increase of 1.92 ± 0.56 in Peak Signal-to-Noise Ratio (PSNR) and 0.023 ± 0.011 in Structural Similarity Index Measure (SSIM). Additionally, we demonstrate DAW-FA’s impact on downstream tasks: Alzheimer’s disease classification and whole-brain segmentation, highlighting its potential clinical relevance.

Detection of fruit tree diseases in natural environments: A novel approach based on stereo camera and deep learning

The occurrence of diseases in orchards has a significant impact on fruit yield and quality. Inspection devices equipped with cameras can effectively replace manual intervention in the process of orchard management by swiftly detecting diseases. However, the images captured by such devices often exhibit a wide vision field and contain a significant amount of extraneous information. This paper presented a method for detecting diseases in natural environments based on binocular cameras and deep learning techniques applied to fruit tree leaf images with a wide visual field. Firstly, the ZED2i binocular camera was utilized to capture image pairs from a long distance, simulating the visual field of an inspection device. These image pairs were then processed using the Unimatch stereo matching algorithm to obtain a disparity map and calculated the corresponding depth map. The depth information was used to create a mask, eliminating irrelevant background information from the images. Secondly, a lightweight disease detection (LDD) model was proposed based on the advanced YOLOv5 framework for detecting pear rust and plum perforation diseases. The backbone network consisted of shuffle channel block, inverted shuffle channel block, and convolutional block attention module, with only one detection head used in the classifier part. The final experiments evaluated the segmentation, model improvement, and disease spots detection performance. The results showed that the depth map obtained using Unimatch for stereo matching was more accurate than that obtained using the ZED software development kit. In ablation experiments, LDD achieved a mean average precision of 93.0%, with a model size of only 3.9 MB, outperforming the original YOLOv5-s model. Preprocessed images with depth information exhibited improved detection performance, achieved a F1 score of 93.62%, which was a 10.92% improvement over direct detection of the original images. Overall, the presented method successfully addresses the issue of background interference when detecting diseases of fruit tree leaf with a wide visual field, providing a technical basis for automated orchard inspection operations.

Image restoration for digital line drawings using line masks

The restoration of digital images holds practical significance due to the fact that degradation of digital image data on the internet is common. State-of-the-art image restoration methods usually employ end-to-end trained networks. However, we argue that a network trained with diverse image pairs is not optimal for restoring line drawings which have extensive plain backgrounds. We propose a line-drawing restoration framework which takes a restoration neural network as backbone and processes an input degraded line drawing in two steps. First, a proposed mask-predicting network predicts a line mask which indicates the possible location of foreground and background in the potential original line drawing. Next, we feed the degraded input line drawing together with the predicted line mask into the backbone restoration network. The traditional L1 loss for the backbone restoration network is substituted with a masked Mean Square Error (MSE) loss. We test our framework on two classical image restoration tasks: JPEG restoration and super-resolution, and experiments demonstrate that our framework can achieve better quantitative and visual results in most cases.