Blurry Regions Research Articles

Automatic Segmentation Model for Parkinson’s Images Based on SA-U2-Net

To effectively solve the problem of a small proportion of substantia nigra and midbrain regions in Magnetic Resolution Imaging (MRI) images of Parkinson’s disease (PD) patients, unclear boundaries with surrounding tissues, and difficulty in accurately delineating boundaries, an improved U2-Net algorithm for Parkinson’s substantia nigra midbrain image segmentation was proposed. This algorithm first improves the Residual U-blocks (RSU) and RSU-4F modules using the Shuffle Attention (SA) module, enhancing the network’s attention to Parkinson’s substantia nigra and midbrain blurry regions in the encoding and decoding layers. Next, replace some ordinary convolutions in RSU-4F with DynamicConv (DyConv), capture local features through dynamic convolution kernels, and improve the segmentation performance of Parkinson’s images. The experimental data used clinical data, and the improved algorithm achieved Dice, Precision, Recall, and mIou of 80.27%, 89.99%, 88.88%, and 82.38% in substantia nigra segmentation, respectively. The experimental results show that this algorithm can achieve more precise segmentation of Parkinson’s images.

Position Fusing and Refining for Clear Salient Object Detection.

Multilevel feature fusion plays a pivotal role in salient object detection (SOD). High-level features present rich semantic information but lack object position information, whereas low-level features contain object position information but are mixed with noises such as backgrounds. Appropriately addressing the gap between low-and high-level features is important in SOD. We first propose a global position embedding attention (GPEA) module to minimize the discrepancy between multilevel features in this article. We extract the position information by utilizing the semantic information at high-level features to resist noises at low-level features. Object refine attention (ORA) module is introduced to refine features used to predict saliency maps further without any additional supervision and heighten discriminative regions near the salient object, such as boundaries. Moreover, we find that the saliency maps generated by the previous methods contain some blurry regions, and we design a pixel value (PV) loss to help the model generate saliency maps with improved clarity. Experimental results on five commonly used SOD datasets demonstrated that the proposed method is effective and outperforms the state-of-the-art approaches on multiple metrics.

Blind Attention Geometric Restraint Neural Network for Single Image Dynamic/Defocus Deblurring.

Based on the information loss analysis of the blur accumulation model, a novel single-image deblurring method is proposed. We apply the recurrent neural network architecture to capture the attention perception map and the generative adversarial network (GAN) architecture to yield the deblurring image. Considering that the attention mechanism has to make hard decisions about specific parts of the input image to be focused on since blurry regions are not given, we propose a new adaptive attention disentanglement model based on the variation blind source separation, which provides the global geometric restraint to reduce the large solution space, so that the generator can realistically restore details on blurry regions, and the discriminator can accurately assess the content consistency of the restored regions. Since we combine blind source separation, attention geometric restraint with GANs, we name the proposed method BAGdeblur. Extensive evaluations on quantitative and qualitative experiments show that the proposed method achieves the state-of-the-art performance on both synthetic datasets and real-world blurry images.

Conditional reiterative High-Fidelity GAN inversion for image editing

Our work introduces a conditional reiteration mechanism for High-Fidelity GAN (Generative Adversarial Networks) inversion (HFGI), preserving image-specific details (like background, appearance, etc.) for both normal and out-of-domain images (e.g. heavy makeup faces). The HFGI encoder’s single-stage conditional latent maps result in blurry regions in restored images and loss of detailed information during editing. To address this, we proposed a reiterative conditional latents method that restores image-specific details sharply. The process involves two stages of iterations, reconstructing the image in the first stage, and refining image-specific details using conditional latent codes in the second stage. Our model successfully inverts out-of-domain images while preserving all details and supports InterfaceGAN, GANspace, and StyleClip for editing. We compare our approach with state-of-the-art GAN inversion methods on FFHQ (Flickr-Faces-HQ) Dataset, demonstrating significant improvements in inversion and editing quality.

Semi-Supervised Medical Image Segmentation With Voxel Stability and Reliability Constraints.

Semi-supervised learning is becoming an effective solution in medical image segmentation because annotations are costly and tedious to acquire. Methods based on the teacher-student model use consistency regularization and uncertainty estimation and have shown good potential in dealing with limited annotated data. Nevertheless, the existing teacher-student model is seriously limited by the exponential moving average algorithm, which leads to the optimization trap. Moreover, the classic uncertainty estimation method calculates the global uncertainty for images but does not consider local region-level uncertainty, which is unsuitable for medical images with blurry regions. In this paper, the Voxel Stability and Reliability Constraint (VSRC) model is proposed to address these issues. Specifically, the Voxel Stability Constraint (VSC) strategy is introduced to optimize parameters and exchange effective knowledge between two independent initialized models, which can break through the performance bottleneck and avoid model collapse. Moreover, a new uncertainty estimation strategy, the Voxel Reliability Constraint (VRC), is proposed for use in our semi-supervised model to consider the uncertainty at the local region level. We further extend our model to auxiliary tasks and propose a task-level consistency regularization with uncertainty estimation. Extensive experiments on two 3D medical image datasets demonstrate that our method outperforms other state-of-the-art semi-supervised medical image segmentation methods under limited supervision. The source code and pre-trained models of this method are available at https://github.com/zyvcks/JBHI-VSRC.

Detection of a new crescent moon using the Maximally Stable Extremal Regions (MSER) technique

Visibility of a new crescent moon is crucial for Muslims when declaring the beginning of the Islamic calendar. For many years, astronomers had observed the visibility of the new crescent moon using various techniques, such as the naked eye and optical instruments. In line with technological developments, astronomers have changed their observation technique to using astronomical imaging with the aid of a digital camera attached to a telescope. This helps to capture and save the images of the new crescent moon. However, the new crescent moon image has presented difficulties to astronomers as it visibility in the image is difficult to detect due to a low contrast with the sky. Hence, the captured image could not be presented as evidence of its visibility. This study proposes the use of Contrast Limited Adaptive Histogram Equalization (CLAHE) for contrast enhancement. Then, the bilateral filter for image denoising was applied and lastly, the Maximally Stable Extremal Regions (MSER) detector was used to detect the new crescent moon region in the image. The result shows that this proposed method has acceptable practical results. The new crescent moon images can be enhanced successfully using CLAHE and the bilateral filter technique even when the new crescent moon is not visible to the naked eye. The new crescent moon region can be well detected with the MSER detector although MSER does not perform well on the blurry region of new crescent moon corner. This technique can help astronomers detect the new crescent moon’s visibility for declaring the beginning month of the Islamic calendar.

An Efficient Defocus Blur Segmentation Scheme Based on Hybrid LTP and PCNN.

The defocus or motion effect in images is one of the main reasons for the blurry regions in digital images. It can affect the image artifacts up to some extent. However, there is a need for automatic defocus segmentation to separate blurred and sharp regions to extract the information about defocus-blur objects in some specific areas, for example, scene enhancement and object detection or recognition in defocus-blur images. The existence of defocus-blur segmentation algorithms is less prominent in noise and also costly for designing metric maps of local clarity. In this research, the authors propose a novel and robust defocus-blur segmentation scheme consisting of a Local Ternary Pattern (LTP) measured alongside Pulse Coupled Neural Network (PCNN) technique. The proposed scheme segments the blur region from blurred fragments in the image scene to resolve the limitations mentioned above of the existing defocus segmentation methods. It is noticed that the extracted fusion of upper and lower patterns of proposed sharpness-measure yields more noticeable results in terms of regions and edges compared to referenced algorithms. Besides, the suggested parameters in the proposed descriptor can be flexible to modify for performing numerous settings. To test the proposed scheme’s effectiveness, it is experimentally compared with eight referenced techniques along with a defocus-blur dataset of 1000 semi blurred images of numerous categories. The model adopted various evaluation metrics comprised of Precision, recall, and F1-Score, which improved the efficiency and accuracy of the proposed scheme. Moreover, the proposed scheme used some other flavors of evaluation parameters, e.g., Accuracy, Matthews Correlation-Coefficient (MCC), Dice-Similarity-Coefficient (DSC), and Specificity for ensuring provable evaluation results. Furthermore, the fuzzy-logic-based ranking approach of Evaluation Based on Distance from Average Solution (EDAS) module is also observed in the promising integrity analysis of the defocus blur segmentation and also in minimizing the time complexity.

Image blur analysis and elimination algorithm for an MTF test target of a space camera.

The high-resolution space camera needs to image the fringe target that meets the sampling frequency in the modulation transfer function test. However, blurry regions often appear in the target image, which seriously affects the visual effect and test efficiency. In this paper, by analyzing the imaging process of the rectangle striated target by a complementary metal-oxide-semiconductor transistor detector, a moiré fringe blurry degradation model of the target image is proposed. According to the degradation model, it is concluded that there are low blurry degradation regions in the image. Taking the low blurry degradation region as the reference information and the region similarity as the weight, the weighted spatial filtering and mean filtering are carried out to eliminate the blurry band in the target image. The experimental results show that compared with bilinear interpolation and guidance filter, this algorithm can effectively remove the image blur caused by the target periodic error and the angle error. The work in this paper provides a practical tool and theoretical support for subsequent space camera tests.

RAID-Net: Region-Aware Image Deblurring Network Under Guidance of the Image Blur Formulation

Image deblurring is a challenging field in computational photography and computer vision. In the deep learning era, deblurring methods boosted with neural networks achieve significant results. However, the existing methods mainly focus on solving specific image deblurring problem, and overlook the origin of the motion blur. In this paper, we revisit how blur occurs, and divide them into three categories, i.e. caused by relative motion between camera and scene, caused by the movement of the object itself and the edges of a blurring image, which may meet discontinuity because of the pixels trajectory sampled outside the image. To address the issues of different blurs in an image, we propose a two-stage neural network for image deblurring named RAID-Net. In order to remove the global blurry region caused by camera movements, we first use a U-shape network to get the coarse deblurred image. Then we leverage an adaptive reasoning module to model the relationship between different blurry regions within one image jointly and remove the other two categories of motion blur. Experiments on two public benchmark datasets demonstrate that our method achieves comparable or better results over the state-of-the-art methods.

Heterogeneous Attention Nested U-Shaped Network for Blur Detection

With the popularity of image sensors in various mobile devices, image blurring caused by hand shaking or out of focus becomes ubiquitous, which deteriorates image quality and poses challenges for vision tasks, including object detection, image classification and image segmentation. Designing an efficient blur detection algorithm which can automatically detect and locate blurred regions becomes necessary. In this letter, we design an end-to-end convolution neural network called heterogeneous attention nested U-shaped network (HANUN) for blur detection. We introduce pyramid pooling into encoders to enhance the feature extraction at different scales and reduce the gradual information loss. Inspired by the nested network design, small U-shaped networks are embedded into our decoders to increase the network depth and promote feature fusion with different receptive field scales. In addition, we incorporate a channel attention mechanism in the proposed network to highlight the informative features for detecting the blurry regions. Experimental results show that HANUN outperforms other state-of-the-art algorithms for blur detection tasks on public datasets and real-world images.

Non-uniform motion deblurring with blurry component divided guidance

Blind image deblurring is a fundamental and challenging computer vision problem, which aims to recover both the blur kernel and the latent sharp image from only a blurry observation. Despite the superiority of deep learning methods in image deblurring have displayed, there still exists a major challenge with various non-uniform motion blur. Previous methods simply take all the image features as the input to the decoder, which handles different degrees (e.g. large blur, small blur) simultaneously, leading to challenges for sharp image generation. To tackle the above problems, we present a deep two-branch network to deal with blurry images via a component divided module, which divides an image into two components based on the representation of blurry degree. Specifically, two component attentive blocks are employed to learn attention maps to exploit useful deblurring feature representations on both large and small blurry regions. Then, the blur-aware features are fed into two-branch reconstruction decoders respectively. In addition, a new feature fusion mechanism, orientation-based feature fusion, is proposed to merge sharp features of the two branches. Both qualitative and quantitative experimental results show that our method performs favorably against the state-of-the-art approaches.

A Novel Defocused Image Segmentation Method Based on PCNN and LBP

The defocus blur concept adds an artistic effect and enables an enhancement in the visualization of image scenery. Moreover, some specialized computer vision fields, such as object recognition or scene restoration enhancement, might need to perform segmentation to separate the blurred and non-blurred regions in partially blurred images. This study proposes a sharpness measure comprised of a Local Binary Pattern (LBP) descriptor and Pulse Coupled Neural Network (PCNN) component used to implement a robust approach for segmenting in-focus regions from out of focus sections in the scene. The proposed approach is very robust in the sense that the parameters of the model can be modified to accommodate different settings. The presented metric exploits the fact that, in general, local patches of the image in blurry regions have less prominent LBP descriptors than non-blurry regions. The proposed approach combines this sharpness measure with the PCNN algorithm; the images are segmented along with clear regions and edges of segmented objects. The proposed approach has been tested on a dataset comprised of 1000 defocused images with eight state-of-the-art methods. Based on a set of evaluation metrics, i.e., precision, recall, and F1-Measure, the results show that the proposed algorithm outperforms previous works in terms of prominent accuracy and efficiency improvement. The proposed approach also uses other evaluation parameters, i.e., Accuracy, Matthews Correlation Coefficient (MCC), Dice Similarity Coefficient (DSC), and Specificity, to assess better the results obtained by our proposal. Moreover, we adopted a fuzzy logic ranking scheme inspired by the Evaluation Based on Distance from Average Solution (EDAS) technique to interpret the defocus segmentation integrity. The experimental outputs illustrate that the proposed approach outperforms the referenced methods by optimizing the segmentation quality and reducing the computational complexity.

Identification and enumeration of cyanobacteria species using a deep neural network

Cell classification and cell counting are essential for the detection, monitoring, forecasting, and management of harmful algae populations. Conventional methods of algae classification and cell counting are known to be time-consuming, labor-intensive, and subjective, depending on the expertise of the observers. The objectives of this study were to classify and quantify five cyanobacteria using the deep learning techniques of a fast regional convolutional neural network (R-CNN) and convolutional neural network (CNN). Water samples taken from the Haman weir of Nakdong River and Baekje weir of the Geum River were observed under the optical microscope. The images captured by the microscope were used to classify cyanobacteria species using the fast R-CNN model. Post-processing of the classified images generated by the model reduced the noises of the cell features, thereby improving the accuracy of the CNN model in quantifying cyanobacteria cells. The distinctive morphological features of the five species were extracted by the fast R-CNN model. This model was able to achieve a reasonable agreement with the manual classification results, yielding average precision (AP) values of 0.929, 0.973, 0.829, 0.890, and 0.890 for Microcystis aeruginosa, Microcystis wesenbergii, Dolichospermum, Oscillatoria, and Aphanizomenon, respectively. The CNN model for the Microcystis species obtained an R2 value of 0.775 and RMSE value of 26 cells for training, and an R2 of 0.854 and RMSE of 23 cells for validation. A minor underestimation and overestimation for a population with <50 cells and >250 cells were observed, respectively, which are due to the overlapping of cells and the presence of blurry regions in the input images. In conclusion, this study was able to demonstrate the reliable performance of cyanobacteria classification and cell counting using deep learning approaches.

BR$^2$Net: Defocus Blur Detection Via a Bidirectional Channel Attention Residual Refining Network

Due to the remarkable potential applications, defocus blur detection, which aims to separate blurry regions from an image, has attracted much attention. Although significant progress has been made by many methods, there are still various challenges that hinder the results, e.g., confusing background areas, sensitivity to the scale and missing the boundary details of the defocus blur regions. To solve these issues, in this paper, we propose a deep convolutional neural network (CNN) for defocus blur detection via a Bi-directional Residual Refining network (BR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Net). Specifically, a residual learning and refining module (RLRM) is designed to correct the prediction errors in the intermediate defocus blur map. Then, we develop a bidirectional residual feature refining network with two branches by embedding multiple RLRMs into it for recurrently combining and refining the residual features. One branch of the network refines the residual features from the shallow layers to the deep layers, and the other branch refines the residual features from the deep layers to the shallow layers. In such a manner, both the low-level spatial details and highlevel semantic information can be encoded step by step in two directions to suppress background clutter and enhance the detected region details. The outputs of the two branches are fused to generate the final results. In addition, with the observation that different feature channels have different extents of discrimination for detecting blurred regions, we add a channel attention module to each feature extraction layer to select more discriminative features for residual learning. To promote further research on defocus blur detection, we create a new dataset with various challenging images and manually annotate their corresponding pixelwise ground truths. The proposed network is validated on two commonly used defocus blur detection datasets and our newly collected dataset by comparing it with 10 other state-of-the-art methods. Extensive experiments with ablation studies demonstrate that BR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Net consistently and significantly outperforms the competitors in terms of both the efficiency and accuracy.

Detecting spatiotemporal irregularities in videos via a 3D convolutional autoencoder

Spatiotemporal irregularities (i.e., the uncommon appearance and motion patterns) in videos are difficult to detect, as they are usually not well defined and appear rarely in videos. We tackle this problem by learning normal patterns from regular videos, while treating irregularities as deviations from normal patterns. To this end, we introduce a 3D fully convolutional autoencoder (3D-FCAE) that is trainable in an end-to-end manner to detect both temporal and spatiotemporal irregularities in videos using limited training data. Subsequently, temporal irregularities can be detected as frames with high reconstruction errors, and irregular spatiotemporal patterns can be detected as blurry regions that are not well reconstructed. Our approach can accurately locate temporal and spatiotemporal irregularities thanks to the 3D fully convolutional autoencoder and the explored effective architecture. We evaluate the proposed autoencoder for detecting irregular patterns on benchmark video datasets with weak supervision. Comparisons with state-of-the-art approaches demonstrate the effectiveness of our approach. Moreover, the learned autoencoder shows good generalizability across multiple datasets.

CaricatureShop: Personalized and Photorealistic Caricature Sketching.

In this paper, we propose the first sketching system for interactively personalized and photorealistic face caricaturing. Input an image of a human face, the users can create caricature photos by manipulating its facial feature curves. Our system first performs exaggeration on the recovered 3D face model, which is conducted by assigning the laplacian of each vertex a scaling factor according to the edited sketches. The mapping between 2D sketches and the vertex-wise scaling field is constructed by a novel deep learning architecture. Our approach allows outputting different exaggerations when applying the same sketching on different input figures in term of their different geometric characteristics, which makes the generated results "personalized". With the obtained 3D caricature model, two images are generated, one obtained by applying 2D warping guided by the underlying 3D mesh deformation and the other obtained by re-rendering the deformed 3D textured model. These two images are then seamlessly integrated to produce our final output. Due to the severe stretching of meshes, the rendered texture is of blurry appearances. A deep learning approach is exploited to infer the missing details for enhancing these blurry regions. Moreover, a relighting operation is invented to further improve the photorealism of the result. These further make our results "photorealistic". The qualitative experiment results validated the efficiency of our sketching system.

Detailed Structures and Formation Mechanisms of Well-Known Al10RE2Mn7 Phase in Die-Cast Mg–4Al–4RE–0.3Mn Alloy

The detailed structures and the corresponding formation mechanisms of the well-known Al10RE2Mn7 phase in the conventional die-cast Mg–4Al–4RE–0.3Mn alloy were thoroughly investigated using transmission electron microscopy (TEM). The results indicate that the Al10RE2Mn7 phase ordinarily contains both normal $$(11\overline{2} 1)$$ twins and orientation twins. Both detailed TEM observations and density functional theory calculations indicate that the Al10RE2Mn7 phase is transferred from the Al8REMn4 phase following an orientation relationship as $$[010]_{{{\text{Al}}_{ 8} {\text{REMn}}_{ 4} }} / /[\overline{1} 101]_{{{\text{Al}}_{ 1 0} {\text{RE}}_{ 2} {\text{Mn}}_{ 7} }}$$ and $$(101)_{{{\text{Al}}_{ 8} {\text{REMn}}_{ 4} }} / /(11\overline{2} 0)_{{{\text{Al}}_{ 1 0} {\text{RE}}_{ 2} {\text{Mn}}_{ 7} }}$$ . Moreover, forming orientation twins in the Al10RE2Mn7 phase is attributed to the blurry regions at incoherent twin boundaries in the Al8REMn4 phase. Finally, these formed orientation twins result in the $$(11\overline{2} 1)$$ twins in the Al10RE2Mn7 phase.

NIQSV+: A No-Reference Synthesized View Quality Assessment Metric.

Benefiting from multi-view video plus depth and depth-image-based-rendering technologies, only limited views of a real 3-D scene need to be captured, compressed, and transmitted. However, the quality assessment of synthesized views is very challenging, since some new types of distortions, which are inherently different from the texture coding errors, are inevitably produced by view synthesis and depth map compression, and the corresponding original views (reference views) are usually not available. Thus the full-reference quality metrics cannot be used for synthesized views. In this paper, we propose a novel no-reference image quality assessment method for 3-D synthesized views (called NIQSV+). This blind metric can evaluate the quality of synthesized views by measuring the typical synthesis distortions: blurry regions, black holes, and stretching, with access to neither the reference image nor the depth map. To evaluate the performance of the proposed method, we compare it with four full-reference 3-D (synthesized view dedicated) metrics, five full-reference 2-D metrics, and three no-reference 2-D metrics. In terms of their correlations with subjective scores, our experimental results show that the proposed no-reference metric approaches the best of the state-of-the-art full reference and no-reference 3-D metrics; and outperforms the widely used no-reference and full-reference 2-D metrics significantly. In terms of its approximation of human ranking, the proposed metric achieves the best performance in the experimental test.

AllFocus: Patch-Based Video Out-of-Focus Blur Reconstruction

Amateur videos always contain focusing issues. A focusing mistake may produce out-of-focus blur, which seriously degrades the expressive force of the video. In this paper, we propose a patch-based method to remove the out-of-focus blur of a video and build an all-in-focus video. We assume that the out-of-focus blurry region in one frame will be clear in a portion of other frames; thus, the clear corresponding regions can be used to reconstruct the blurry one. We divide each video frame into a grid of patches and track each patch in the surrounding frames. We independently reconstruct each video frame by building a Markov random field model to identify the optimal target patches that are sharp, similar to the original patches, and are coherent with their neighboring patches within the overlapped regions. To recover an all-in-focus video, an iterative framework is utilized, in which the reconstructed video of each iteration is substituted in the next iteration. Finally, we employ the idea of a bilateral filter to temporally smooth the reconstructed video. The experimental results and the comparison with the previous works demonstrate the effectiveness of our method.

LBP-Based Segmentation of Defocus Blur.

Defocus blur is extremely common in images captured using optical imaging systems. It may be undesirable, but may also be an intentional artistic effect, thus it can either enhance or inhibit our visual perception of the image scene. For tasks, such as image restoration and object recognition, one might want to segment a partially blurred image into blurred and non-blurred regions. In this paper, we propose a sharpness metric based on local binary patterns and a robust segmentation algorithm to separate in- and out-of-focus image regions. The proposed sharpness metric exploits the observation that most local image patches in blurry regions have significantly fewer of certain local binary patterns compared with those in sharp regions. Using this metric together with image matting and multi-scale inference, we obtained high-quality sharpness maps. Tests on hundreds of partially blurred images were used to evaluate our blur segmentation algorithm and six comparator methods. The results show that our algorithm achieves comparative segmentation results with the state of the art and have big speed advantage over the others.