Missing Regions Research Articles

Missing Data Reconstruction Based on Spectral k-Support Norm Minimization for NB-IoT Data

An effective fraction of data with missing values from various physiochemical sensors in the Internet of Things is still emerging owing to unreliable links and accidental damage. This phenomenon will limit the predicative ability and performance for supporting data analyses by IoT-based platforms. Therefore, it is necessary to exploit a way to reconstruct these lost data with high accuracy. A new data reconstruction method based on spectral k-support norm minimization (DR-SKSNM) is proposed for NB-IoT data, and a relative density-based clustering algorithm is embedded into model processing for improving the accuracy of reconstruction. First, sensors are grouped by similar patterns of measurement. A relative density-based clustering, which can effectively identify clusters in data sets with different densities, is applied to separate sensors into different groups. Second, based on the correlations of sensor data and its joint low rank, an algorithm based on the matrix spectral k-support norm minimization with automatic weight is developed. Moreover, the alternating direction method of multipliers (ADMM) is used to obtain its optimal solution. Finally, the proposed method is evaluated by using two simulated and real sensor data sources from Panzhihua environmental monitoring station with random missing patterns and consecutive missing patterns. From the simulation results, it is proved that our algorithm performs well, and it can propagate through low-rank characteristics to estimate a large missing region’s value.

A Partial Differential Equation-Based Image Restoration Method in Environmental Art Design

With the rapid development of networks and the emergence of various devices, images have become the main form of information transmission in real life. Image restoration, as an important branch of image processing, can be applied to real-life situations such as pixel loss in image transmission or network prone to packet loss. However, existing image restoration algorithms have disadvantages such as fuzzy restoration effect and slow speed; to solve such problems, this paper adopts a dual discriminator model based on generative adversarial networks, which effectively improves the restoration accuracy by adding local discriminators to track the information of local missing regions of images. However, the model is not optimistic in generating reasonable semantic information, and for this reason, a partial differential equation-based image restoration model is proposed. A classifier and a feature extraction network are added to the dual discriminator model to provide category, style, and content loss constraints to the generative network, respectively. To address the training instability problem of discriminator design, spectral normalization is introduced to the discriminator design. Extensive experiments are conducted on a data dataset of partial differential equations, and the results show that the partial differential equation-based image restoration model provides significant improvements in image restoration over previous methods and that image restoration techniques are exceptionally important in the application of environmental art design.

Restoring SAR Images Using Effective Image Restoration Approach

Landsat 7 Enhanced Thematic Mapper Plus satellite images presents an important data source for many applications related to remote sensing. An effective image restoration method is proposed to fill the missing information in the satellite images. The segmentation of satellite images to find the SLIC Super pixels and then to find the image Segments. The Boundary Reconstruction is performed using Edge Matching to find the area of the missing region. Peak Signal to Noise Ratio and Root Mean Square Error using with boundary reconstruction and without boundary reconstruction to evaluate the quality and the error rate of the satellite images. The results show the capability to predict the missing values accurately in terms of quality, time without need of external information.The values for PSNR has changed from 25 to 90 and RMSE has changed from 180 to 4 in Red Channel of an image.This indicates that quality of the image is high and error rate is less.

Hole-filling based on content loss indexed 3D partial convolution network for freehand ultrasound reconstruction

Background and objectiveDuring the 3D reconstruction of ultrasound volume from 2D B-scan ultrasound images, holes are usually found in the reconstructed 3D volumes due to the fast scans. This condition will affect the positioning and judgment of the doctor to the lesion. Hence, in this study, we propose to fill the holes by using a novel content loss indexed 3D partial convolution network for 3D freehand ultrasound volume reconstruction. The network can synthesize novel ultrasound volume structures and reconstruct ultrasound volume with missing regions with variable sizes and at arbitrary locations. MethodsFirst, the 3D partial convolution is introduced into the convolutional layer, which is masked and renormalized to be conditioned on only valid voxels. Then, the mask in the next layer is automatically updated as a part of the forward pass. To better preserve texture and structure details of the reconstruction results, we couple the adversarial loss of the least squares generative adversarial network (LSGAN) with the innovative content loss, which consists of the context loss, the feature-matching loss and the total variation loss. Thereafter, we introduce a novel spectral-normalized LSGAN by adding spectral normalization (SN) to the generator and discriminator of the LSGAN. The proposed method is simple in formulation, and is stable in training. ResultsExperiments on public and in-vivo ultrasound datasets and comparisons with popular algorithms demonstrate that the proposed approach can generate high-quality hole-filling results with preserved perceptual image details. ConclusionsConsidering the high quality of the hole-filling results, the proposed method can effectively fill the missing regions in the reconstructed 3D ultrasound volume from 2D ultrasound image sequences.

A Context-Based Multi-Scale Discriminant Model for Natural Image Inpainting

Image inpainting is a classical yet challenging problem in computer vison with many real-world applications. In this paper, we propose a novel method for inpainting based on Convolutional Neural Network (CNN) and Generative Adversarial Networks (GAN). By analogy with the autoencoder, a new context encoder is proposed to generate the contents of a missing image region conditioned on its surroundings. To improve the stability and efficiency, we also modify the architecture of the context encoder by introducing 1*1 convolution, as well as other improvements. In addition, a multi-scale discriminator combining with GAN is presented. During training, the reconstruction and adversarial losses are used for the global image. To generate more details, we further add a local texture loss for the missing part. Qualitative experiments show that our model performs favorably against classical methods in generating visually plausible inpainting results. Moreover, quantitative experiments substantiate the effectiveness of the proposed method in natural image inpainting.

Therapeutic effects of moxibustion on neonatal mice with hypoxia-ischemia brain injury

Objective: To investigate the effects of moxibustion on the behavioral performance, brain morphological structure of mice with hypoxia-ischemia brain injury and to explore its mechanisms. Methods: One hundred and six ICR mice were randomly divided into three groups, sham group (n=23), model group (n=46) and moxibustion-treated group (n=37). Neonatal hypoxic-ischemia brain injury was induced by ligation of common carotid artery followed by hypoxia (8% oxygen, 100 min), and pups in the moxibustion-treated group were administered suspended moxibustion on the Dazhui points (GV14) at a height of approximately 2 cm over a hairless area of the skin once a day for 4 days (i.e. at 2, 24, 48 and 72 hours after hypoxia-ischemia procedure). Behavioral tests were used to evaluate behavioral performance. HE staining was used to observe brain morphological structure. Western blot was used to detect the expression of SOD2 protein, and spectrophotometry was used to determine the content of MDA in the ipsilateral brain. Results: Mouse pups in sham group showed that the behavioral performance was normal, the brain tissue cells were densely and neatly arranged, the expression of SOD2 and the level of MDA in the brain tissues were normal. Compared with sham group, mouse pups in the HI model group exhibited a significant longer latency to complete the righting reflex, geotaxis reflex, cliff avoidance (P＜0.05) and a marked shorter latency to complete the grip test (P＜0.05); and the HI model group had dramatic brain morphological changes showing missing regions, decreased expression of SOD2 protein (P＜0.05) and increased level of MDA in the brain. Compared with HI model group, mouse pups in the moxibustion-treated group exhibited a significant shorter latency to complete the righting reflex, geotaxis reflex, cliff avoidance test (P＜0.05) and a marked longer latency to complete the grip test (P＜0.05); and the moxibustion-treated group had less brain morphological changes, increased expression of SOD2 protein (P＜0.05) and decreased level of MDA in the brain (P＜0.05) . Conclusion: Moxibustion could improve behavioral performance and attenuate hypoxia-ischemia brain injury, which might be related to increasing the expression of SOD2 protein and decreasing the content of MDA, thus enhancing the anti-oxidative ability.

Medical image inpainting with edge and structure priors

Image inpainting techniques are widely applied to repair distorted or missing regions in the image. However, unreasonable results are usually generated due to lack of structural information. In order to repair the structure rationally, we propose a model based on edge and structure information for medical image inpainting (ESMII). In addition, for overcoming the degradation problem of the deep network, multi-scale residual blocks (MRB) are used to extract features from the image at three scales. Meanwhile, the multi-level loss is proposed to motivate the network synthesizing more significant structural textures. Experimental results on COVID CT, Abdominal CT and Abdominal MRI datasets indicate that ESMII can generate more reasonable structures and fine textures compared to other models. Furthermore, ablation studies for justification verify the effectiveness of different modules in the network. We show that our approach can effectively solve the distortion of medical images.

Detail-enhanced image inpainting based on discrete wavelet transforms

Deep-learning-based method has made great breakthroughs in image inpainting by generating visually plausible contents with reasonable semantic meaning. However, existing deep learning methods still suffer from distorted structures or blurry textures. To mitigate this problem, completing semantic structure and enhancing textural details should be considered simultaneously. To this end, we propose a two-parallel-branch completion network, where the first branch fills semantic content in spatial domain, and the second branch helps to generate high-frequency details in wavelet domain. To reconstruct an inpainted image, the output of the first branch is also decomposed by discrete wavelet transform, and the resulting low-frequency wavelet subband is used jointly with the output of the second branch. In addition, for improving the network capability in semantic understanding, a multi-level fusion module (MLFM) is designed in the first branch to enlarge the receptive field. Furthermore, drawing lessons from some traditional exemplar-based inpainting methods, we develop a free-form spatially discounted mask (SD-mask) to assign different importance priorities for the missing pixels based on their positions, enabling our method to handle missing regions with arbitrary shapes. Extensive experiments on several public datasets demonstrate that the proposed approach outperforms current state-of-the-art ones. The codes are public available at https://github.com/media-sec-lab/DWT_Inpainting.

Experimental velocity data estimation for imperfect particle images using machine learning

We propose a method using supervised machine learning to estimate velocity fields from particle images having missing regions due to experimental limitations. As a first example, a velocity field around a square cylinder at the Reynolds number of ReD=300 is considered. To train machine learning models, we utilize artificial particle images (APIs) as the input data, which mimic the images of the particle image velocimetry (PIV). The output data are the velocity fields, and the correct answers for them are given by a direct numerical simulation (DNS). We examine two types of the input data: APIs without missing regions (i.e., full APIs) and APIs with missing regions (lacked APIs). The missing regions in the lacked APIs are assumed following the exact experimental situation in our wind tunnel setup. The velocity fields estimated from both full and lacked APIs are in great agreement with the reference DNS data in terms of various statistical assessments. We further apply these machine learned models trained with the DNS data to experimental particle images so that their applicability to the exact experimental situation can be investigated. The velocity fields estimated by the machine learned models contain approximately 40 fold denser data than that with the conventional cross-correlation method. This finding suggests that we may be able to obtain finer and hidden structures of the flow field, which cannot be resolved with the conventional cross-correlation method. We also find that even the complex flow structures are hidden due to the alignment of two square cylinders, the machine learned model is able to estimate the field in the missing region reasonably well. The present results indicate a great potential of the proposed machine learning-based method as a new data reconstruction method for PIV.

Single Image Cloud Removal Using U-Net and Generative Adversarial Networks

Cloud removal is a ubiquitous and important task in remote sensing image processing, which aims at restoring the ground regions shadowed by clouds. It is challenging to remove the clouds for a single satellite image due to the difficulty of distinguishing clouds from white objects on the ground and filling the irregular missing regions with visual consistency. In this article, we propose a novel two-stage cloud removal method. The first stage is cloud segmentation, i.e., extracting the clouds and removing the thin clouds directly using U-Net. The second stage is image restoration, i.e., removing the thick cloud and recovering the corresponding irregular missing regions using generative adversarial network (GAN). We evaluate the proposed scheme on both synthetic images and real satellite images (over <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$20\,000\, \times \,20\,000$ </tex-math></inline-formula> pixels). On synthetic images for cloud coverage less than 40%, the proposed scheme achieves improvements of 0.049–0.078 in Structural SIMilarity (SSIM) and 3.8–6.2 dB in peak signal-to-noise ratio (PSNR), while the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> -norm error reduces by 49%–78%, compared with a state-of-the-art deep learning method Pix2Pix. On real satellite images, we demonstrate the consistent visual results of the proposed scheme.

Prosthodontic Management of Single Anterior Missing Tooth with Immediate Loading Implant Using Leukocyte-Platelet-Rich Fibrin

AbstractImmediate loading of dental implant in the anterior tooth region has been a growing trend and eliminates the compromised esthetics, function, discomfort, and inconvenience associated with traditional two-stage implant procedures. This case report presents the clinical application of implant placement with immediate loading using leukocyte-platelet-rich fibrin (L-PRF) in the anterior esthetic region.The implant was planned to rehabilitate in the anterior missing tooth region. As per the patient's desire, the implant was immediately loaded and L-PRF membrane was placed around the implant under local anesthesia on the day of surgery.The use of L-PRF for the maintenance of crestal bone and soft tissue at the implant sites provided an adequate clinical condition for better esthetics.

Towards point cloud completion: Point Rank Sampling and Cross-Cascade Graph CNN

The Point Fractal Network (PF-Net) is a seminal work with capability of completing the missing regions of point clouds. However, the multi-resolution structure of PF-Net neglects effective feature fusion between each resolution, which makes the resulting shape lack local geometric details. To tackle this problem, we design a novel shape completion network named PRSCN. We first present Point Rank Sampling to rate and sample feature points more objectively through local outline form. In this way, the sampled points can facilitate the downstream tasks. Subsequently, considering the correlation between features from different scales, we design a Cross-Cascade Module to combine features hierarchically. Moreover, we propose Leap-type EdgeConv to enlarge the receptive field while maintain the kernel size unchanged. These improvements together make our CD error 3% lower than that of state-of-the-art method on ShapeNet-part dataset.

Feature Recognition of Abstract Art Painting Multilevel Based Convolutional Ancient Recognition Neural Network Method

Art can be used to spread an inspirational message to encourage people and accomplish great things in life. The art of communication among people can be a mode of communication to concentrate on common issues to improve humankind. The challenging characteristics in art painting include such as degradation, cracking, and flaking is considered an essential factor. In this paper, the Multilevel Based Convolutional Ancient Recognition Neural Network Method (M-CARNNM) has been proposed to incorporate experts’ suggestions, helping artists envisage how the ancient painting may have looked after restoration. Mid-frequency analysis is introduced to reinforce the rough estimation of complete images by adding missing regions and the nearest neighbor pixels to match the maps. A domain-specific pyramid network is used to capture various space context amounts. Experimental results effectively predict the proposed method for large areas of lack of information and produce controllable vector graphics, photographic painting, and high-frequency results.

Fast Generation of High-Fidelity RGB-D Images by Deep Learning With Adaptive Convolution

Using the raw data from consumer-level RGB-D cameras as input, we propose a deep-learning-based approach to efficiently generate RGB-D images with completed information in high resolution. To process the input images in low resolution with missing regions, new operators for adaptive convolution are introduced in our deep-learning network that consists of three cascaded modules—the completion module, the refinement module, and the super-resolution module. The completion module is based on an architecture of encoder–decoder, where the features of input raw RGB-D will be automatically extracted by the encoding layers of a deep neural network. The decoding layers are applied to reconstruct the completed depth map, which is followed by a refinement module to sharpen the boundary of different regions. For the super-resolution module, we generate RGB-D images in high resolution by multiple layers for feature extraction and a layer for upsampling. Benefited from the adaptive convolution operators proposed in this article, our results outperform the existing deep-learning-based approaches for RGB-D image complete and super-resolution. As an end-to-end approach, high-fidelity RGB-D images can be generated efficiently at the rate of 22 frames/s. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —With the development of consumer-level RGB-D cameras, industries have started to employ these low-cost sensors in many robotic and automation applications. However, images generated by consumer-level RGB-D cameras are generally in low resolution. Moreover, the depth images often have incomplete regions when the surface of an object is transparent, highly reflective, or beyond the distance of sensing. With the help of our method, engineers are able to “repair” the images captured by consumer-level RGB-D cameras in high efficiency. As the typical deep-learning networks are employed in this approach, the proposed approach fits well with the GPU-based hardware architecture of deep-learning computation—therefore, it potentially can be integrated into the hardware of RGB-D cameras.

Region Filling with Super Resolution Algorithm

Region filling which has another name inpainting, is an approach to find the values of missing pixels from data available in the remaining portion of the image. The missing information must be recalculated in a distinctly convincing manner, such that, image look seamless. This research work has built a methodology for completely automating patch priority based region filling process. To reduce the computational time, low resolution image is constructed from input image. Based on texel of an image, patch size is determined. Several low resolution image with missing region filled is generated using region filling algorithm. Pixel information from these low resolution images is consolidated to produce single low resolution region filled image. Finally, super resolution algorithm is applied to enhance the quality of image and regain all specifics of image. This methodology of identifying patch size based on input fed has an advantage over filling algorithms which in true sense automate the process of region filling, to deal with sensitivity in region filling, algorithm different parameter settings are used and functioning with coarse version of image will notably reduce the computational time.

Unsupervised Shape Completion via Deep Prior in the Neural Tangent Kernel Perspective

We present a novel approach for completing and reconstructing 3D shapes from incomplete scanned data by using deep neural networks. Rather than being trained on supervised completion tasks and applied on a testing shape, the network is optimized from scratch on the single testing shape to fully adapt to the shape and complete the missing data using contextual guidance from the known regions. The ability to complete missing data by an untrained neural network is usually referred to as the deep prior . In this article, we interpret the deep prior from a neural tangent kernel (NTK) perspective and show that the completed shape patches by the trained CNN are naturally similar to existing patches, as they are proximate in the kernel feature space induced by NTK. The interpretation allows us to design more efficient network structures and learning mechanisms for the shape completion and reconstruction task. Being more aware of structural regularities than both traditional and other unsupervised learning-based reconstruction methods, our approach completes large missing regions with plausible shapes and complements supervised learning-based methods that use database priors by requiring no extra training dataset and showing flexible adaptation to a particular shape instance.

Face inpainting based on GAN by facial prediction and fusion as guidance information

Face inpainting, a special case of image inpainting, aims to complete the occluded facial regions with unconstrained pose and orientation. However, existing methods generate unsatisfying results with easily detectable flaws. There are often fuzzy boundaries and details near the holes. Especially, for face inpainting, the face region semantic information (face structure, contour, and content information) has not been fully utilized, which leads to unnatural face images, such as asymmetry eyebrow and different sizes of eyes. This is unrealistic in many practical applications. To solve the problems, a new generative adversarial network by facial prediction and fusion as guidance information, is proposed for large missing regions of face inpainting. In the proposed method, two stages are adopted to complete coarse inpainting and refinement of the face. In Stage-I, we combine generator with a new encoder–decoder network with variational autoencoder-based backbone to predict the face region semantic information (including face structure, contour and content information) and do facial fusion for face inpainting. This could fully explore face region semantic information and generate coordinated coarse face images. Stage-II builds upon Stage-I results to refine face image. Both global and patch discriminators are used to synthesize high-quality photo-realistic inpainting. Experimental results on both CelebA and CelebA-HQ datasets demonstrate the effectiveness and efficiency of our method.

Image inpainting network for filling large missing regions using residual gather

Filling arbitrary missing regions with visually plausible contents is a challenging task. Existing methods have accomplished promising performance for inpainting regular or small defects. However, filling large continuous holes is still difficult due to out of step with the performance requirement for inferring large region contents. Considering that the model needs to predict more missing contents with less information when filling large holes, we first introduce a Residual Gather Module (RGM) for high prediction performance by fully taking advantage of residual learning and capturing feature information from a hierarchical distribution. To obtain both high-quality textures and deep model performance, we propose a novel image translation architecture. Different from U-Net, our skip method is semi-complete form. It eliminates losing details caused by downsampling and is capable of designing very deep generative models. Experiments demonstrate that the proposed method exhibits outstanding performance on filling large missing regions.

Thickness accuracy of virtually designed patient-specific implants for large neurocranial defects.

The combination of computer‐aided design (CAD) techniques based on computed tomography (CT) data to generate patient‐specific implants is in use for decades. However, persisting disadvantages are complicated design procedures and rigid reconstruction protocols, for example, for tailored implants mimicking the patient‐specific thickness distribution of missing cranial bone. In this study we used two different approaches, CAD‐ versus thin‐plate spline (TPS)‐based implants, to reconstruct extensive unilateral and bilateral cranial defects in three clinical cases. We used CT data of three complete human crania that were virtually damaged according to the missing regions in the clinical cases. In total, we carried out 132 virtual reconstructions and quantified accuracy from the original to the generated implant and deviations in the resulting implant thickness as root‐mean‐square error (RMSE). Reconstructions using TPS showed an RMSE of 0.08–0.18 mm in relation to geometric accuracy. CAD‐based implants showed an RMSE of 0.50–1.25 mm. RMSE in relation to implant thickness was between 0.63 and 0.70 mm (TPS) while values for CAD‐based implants were significantly higher (0.63–1.67 mm). While both approaches provide implants showing a high accuracy, the TPS‐based approach additionally provides implants that accurately reproduce the patient‐specific thickness distribution of the affected cranial region.

RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy

Colonoscopy is the gold standard for pre-cancerous polyps screening and treatment. The polyp detection rate is highly tied to the percentage of surveyed colonic surface. However, current colonoscopy technique cannot guarantee that all the colonic surface is well examined because of incomplete camera orientations and of occlusions. The missing regions can hardly be noticed in a continuous first-person perspective. Therefore, a useful contribution would be an automatic system that can compute missing regions from an endoscopic video in real-time and alert the endoscopists when a large missing region is detected. We present a novel method that reconstructs dense chunks of a 3D colon in real time, leaving the unsurveyed part unreconstructed. The method combines a standard SLAM system with a depth and pose prediction network to achieve much more robust tracking and less drift. It addresses the difficulties for colonoscopic images of existing simultaneous localization and mapping (SLAM) systems and end-to-end deep learning methods.