Missing Regions Research Articles

388 Young Scholar Award Talk: Increasing Efficiency of Finishing Beef Production: Optimizing Starch Digestion, Improving Flaking Quality, and Reducing Methane Emissions

Abstract The objectives of my dissertation research are to develop strategies to increase the efficiency of finishing beef production by optimizing the site and extent of starch digestion, improving flaking quality and consistency, and reducing methane emissions. Post-flaking sampling and handling techniques such as sifting steam-flaked corn samples with a sieve (P < 0.01), air equilibration time (P < 0.01), and storage temperature during shipping (P < 0.01) all influenced enzymatic starch availability of steam-flaked corn. Inducing starch retrogradation of steam-flaked corn decreased the rate (P < 0.01), but not the extent (P = 0.80), of ruminal dry matter degradation and thus, energetic efficiency was predicted to increase by up to 3.4% because of increased postruminal flow and digestion of starch in the small intestine. If producing steam-flaked corn is not feasible for smaller capacity feedlots, finely ground corn has a similar ruminal starch solubility to steamflaked corn (38.1% vs. 44.4, respectively, P > 0.05), suggesting an alternative processing method to increase feeding value in finishing diets. Duodenal infusion of glucoamylase (fungal origin) increased (P = 0.02) small intestinal starch disappearance in steers, while duodenal infusion of porcine α-amylase did not (P = 0.41), supporting the hypothesis that small intestinal carbohydrases are the primary hydrolytic limit to small intestinal starch digestion. Comparative sequencing analyses revealed a missing region of the stalk region of the sucrase-isomaltase protein sequence (AA 47-60) which is important for intracellular sorting of the sucrase subunit to the apical membrane. The absence of this region of the protein likely explains the absence of intestinal sucrase activity and quantitative limitations of starch digestion in the ruminant small intestine. Feeding decoquinate with Yucca schidigera extract for 7-d decreased (P < 0.01) ruminal acetate proportion and tended to increase ruminal (P = 0.08) propionate proportion, suggesting potential for hydrogen shifts which could contribute to reductions in methane emissions with longer feeding periods. The findings of my dissertation research provide: 1) post-flaking sampling and handling recommendations for commercial feedlots and laboratories to improve estimates of starch availability and flaking consistency, 2) a mechanistic target (sucrase-isomaltase) for limitations in small intestinal starch digestion, 3) practical feeding strategies to shift the site of starch digestion to the small intestine to increase feed efficiency (retrogradation), 4) new processing alternatives for small feedlot producers (fine ground corn inclusion), and 5) new information on how combinations of feed additives could potentially decrease methane production in feedlot cattle.

DcTr: Noise-robust point cloud completion by dual-channel transformer with cross-attention

Current point cloud completion research mainly utilizes the global shape representation and local features to recover the missing regions of 3D shape for the partial point cloud. However, these methods suffer from inefficient utilization of local features and unstructured points prediction in local patches, hardly resulting in a well-arranged structure for points. To tackle these problems, we propose to employ Dual-channel Transformer and Cross-attention (CA) for point cloud completion (DcTr). The DcTr is apt at using local features and preserving a well-structured generation process. Specifically, the dual-channel transformer leverages point-wise attention and channel-wise attention to summarize the deconvolution patterns used in the previous Dual-channel Transformer Point Deconvolution (DCTPD) stage to produce the deconvolution in the current DCTPD stage. Meanwhile, we employ cross-attention to convey the geometric information from the local regions of incomplete point clouds for the generation of complete ones at different resolutions. In this way, we can generate the locally compact and structured point cloud by capturing the structure characteristic of 3D shape in local patches. Our experimental results indicate that DcTr outperforms the state-of-the-art point cloud completion methods under several benchmarks and is robust to various kinds of noise.

Deep learning for image inpainting: A survey

Image inpainting has been widely exploited in the field of computer vision and image processing. The main purpose of image inpainting is to produce visually plausible structure and texture for the missing regions of damaged images. In the past decade, the success of deep learning has brought new opportunities to many vision tasks, which promoted the development of a large number of deep learning-based image inpainting methods. Although these methods have many similarities, they also have their own characteristics due to the differences in data types, application scenarios, computing platforms, etc. It is necessary to classify and summarize these methods to provide a reference for the research community. In this survey, we present a comprehensive overview of recent advances in deep learning-based image inpainting. First, we categorize the deep learning-based techniques from multiple perspectives: inpainting strategies, network structures, and loss functions. Second, we summarize the open source codes and representative public datasets, and introduce the evaluation metrics for quantitative comparisons. Third, we summarize the real-world applications of image inpainting in different scenarios, and give a detailed analysis on the performance of different inpainting algorithms. At last, we conclude the survey and discuss about the future directions.

3D terrain estimation from a single landscape image

AbstractThis article presents the first technique to estimate a 3D terrain model from a single landscape image. Although monocular depth estimation also offers single‐image 3D reconstruction, it assigns depth only to pixels visible in the input image, resulting in an incomplete 3D terrain output. Our method generates a complete 3D terrain model as a textured height map via a three‐stage framework using deep neural networks. First, to exploit the performance of pixel‐aligned estimation, we estimate terrain's per‐pixel depth and color free from shadows or lights in the perspective view. Second, we triangulate the RGB‐D data generated in the first stage and rasterize the triangular mesh from the top view to obtain an incomplete textured height map. Finally, we inpaint the depth and color in the missing regions. Because there are many possible ways to complete the missing regions, we synthesize diverse shapes and textures during inpainting using a variational autoencoder. Qualitative and quantitative experiments reveal that our method outperforms existing methods applying a direct perspective‐to‐top view transform as image‐to‐image translation.

GLF-CR: SAR-enhanced cloud removal with global–local fusion

The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degeneration. In this paper, we propose a novel global–local fusion based cloud removal (GLF-CR) algorithm to leverage the complementary information embedded in SAR images. Exploiting the power of SAR information to promote cloud removal entails two aspects. The first, global fusion, guides the relationship among all local optical windows to maintain the structure of the recovered region consistent with the remaining cloud-free regions. The second, local fusion, transfers complementary information embedded in the SAR image that corresponds to cloudy areas to generate reliable texture details of the missing regions, and uses dynamic filtering to alleviate the performance degradation caused by speckle noise. Extensive evaluation demonstrates that the proposed algorithm can yield high quality cloud-free images and outperform state-of-the-art cloud removal algorithms with a gain about 1.7 dB in terms of PSNR on SEN12MS-CR dataset.

A Pixel-Wise Foreign Object Debris Detection Method Based on Multi-Scale Feature Inpainting

In the aviation industry, foreign object debris (FOD) on airport runways is a serious threat to aircraft during takeoff and landing. Therefore, FOD detection is important for improving the safety of aircraft flight. In this paper, an unsupervised anomaly detection method called Multi-Scale Feature Inpainting (MSFI) is proposed to perform FOD detection in images, in which FOD is defined as an anomaly. This method adopts a pre-trained deep convolutional neural network (CNN) to generate multi-scale features for the input images. Based on the multi-scale features, a deep feature inpainting module is designed and trained to learn how to reconstruct the missing region masked by the multi-scale grid masks. During the inference stage, an anomaly map for the test image is obtained by computing the difference between the original feature and its reconstruction. Based on the anomaly map, the abnormal regions are identified and located. The performance of the proposed method is demonstrated on a newly collected FOD dataset and the public benchmark dataset MVTec AD. The results show that the proposed method is superior to other methods.

Cosmos Propagation Network: Deep learning model for point cloud completion

Point clouds measured by 3D scanning devices often have partially missing data due to the view positioning of the scanner. The missing data can reduce the performance of a point cloud in downstream tasks such as segmentation, location, and pose estimation. Consequently, 3D point cloud completion aims to predict the missing regions of incomplete objects for these fundamental 3D vision tasks. However, predicting the complete object can easily diminish the detail or structure of a measured region, which usually does not require repair. This study proposes a novel neural network architecture, Cosmos Propagation Network (CP-Net), for 3D point cloud completion. CP-Net extracts latent features in different scales from incomplete point clouds used as input. For point cloud generation, we propose a novel point expand method using a Mirror Expand module. Compared with existing methods, our Mirror Expand module introduces less information redundancy, which makes the distribution of points more reliable. CP-Net predicts the details of missing regions and maintains a clear general structure. The performance of CP-Net on several benchmarks was compared to that of current baseline methods. Compared to the existing methods, CP-Net showed the best performance for various metrics. Thus, CP-Net is expected to help address various problems related to 3D point cloud completion. Its source code is available at https://github.com/ark1234/CP-Net.

Mapping of PET/CT-based regional nodes distribution of recurrent/advanced breast cancer and comparison with current delineation atlas.

To localize the distribution of regional nodes in recurrent/advanced breast cancer patients based on 18-fludeoxyglucose (FDG) positron emission tomography/CT (PET/CT) images and validate the coverage of clinical target volumes (CTVs) for regional nodes with current contouring guidelines. We enrolled 154 recurrent/advanced breast cancer patients with FDG-avid regional nodes who underwent PET/CT between January 2018 and June 2020. Involvement of lymph node regions including axillary lymph node level I-III (ALN-I, ALN-II, ALN-III), Rotter's nodes (RN), medial supraclavicular (SC-M), lateral supraclavicular (SC-L) and internal mammary nodes (IMN) was recorded respectively. Coverage of the CTVs in different atlases and the locations of out-of-field were evaluated. A total of 348 lymph node regions containing disease were identified, including ALN-I 109, ALN-II 46, ALN-III 36, RN 17, SC-M 68, SC-L 36 and IMN 36. Recurrent ALNs mainly located cranially and ventrally to the axillary vein (AV). Ipsilateral cervical nodes were simultaneously affected in 33/76 SC positive patients. RADCOMP (306/348) and RUIJIN (291/348) guidelines had higher coverage compared with RTOG (205/348) and ESTRO (202/348) guidelines (p < 0.001, respectively). In primary non-metastastic and recurrent patients, major missings located in SC-L (7/7, 17/17) and IMN (7/10, 15/19) for RTOG guideline while SC-L (7/7, 17/17) for ESTRO guideline (p < 0.001, respectively). Among recurrent patients, SC-M (22/31) was another major missing area for ESTRO guideline (p < 0.001). The current guidelines effectively cover most regional nodes in postoperative breast cancer patients. SC-L and IMN were the major missing regions. Recurrent ALNs were most often seen in cranial and ventral to the AV. The CTV of patients with clinically positive SC was recommended to extend up to the hyoid level. The CTVs should be adjusted based on risks of recurrence individually. The difference of regional nodes delineation between current guidelines mainly located in SC and IMN regions. High axilla including subclavicular nodes and the RN above AV for recurrent patients and the region between cricoid and hyoid for positive SC patients should be meticulously contoured.

Features of smaller ribosomes in candidate phyla radiation (CPR) bacteria revealed with a molecular evolutionary analysis.

The candidate phyla radiation (CPR) is a large bacterial group consisting mainly of uncultured lineages. They have small cells and small genomes, and they often lack ribosomal proteins uL1, bL9, and/or uL30, which are basically ubiquitous in non-CPR bacteria. Here, we comprehensively analyzed the genomic information on CPR bacteria and identified their unique properties. The distribution of protein lengths in CPR bacteria peaks at around 100–150 amino acids, whereas the position of the peak varies in the range of 100–300 amino acids in free-living non-CPR bacteria, and at around 100–200 amino acids in most symbiotic non-CPR bacteria. These results show that the proteins of CPR bacteria are smaller, on average, than those of free-living non-CPR bacteria, like those of symbiotic non-CPR bacteria. We found that ribosomal proteins bL28, uL29, bL32, and bL33 have been lost in CPR bacteria in a taxonomic lineage-specific manner. Moreover, the sequences of approximately half of all ribosomal proteins of CPR differ, in part, from those of non-CPR bacteria, with missing regions or specifically added regions. We also found that several regions in the 16S, 23S, and 5S rRNAs of CPR bacteria are lacking, which presumably caused the total predicted lengths of the three rRNAs of CPR bacteria to be smaller than those of non-CPR bacteria. The regions missing in the CPR ribosomal proteins and rRNAs are located near the surface of the ribosome, and some are close to one another. These observations suggest that ribosomes are smaller in CPR bacteria than those in free-living non-CPR bacteria, with simplified surface structures.

Synchronization of turbulence in channel flow

Synchronization of turbulence in channel flow is investigated using continuous data assimilation. The flow is unknown within a region of the channel. Beyond this region the velocity field is provided, and is directly prescribed in the simulation, while the pressure is unknown throughout the entire domain. Synchronization takes place when the simulation recovers the full true state of the flow, or in other words when the missing region is accurately re-established, spontaneously. Successful synchronization depends on the orientation, location and size of the missing layer. For friction Reynolds numbers up to one thousand, wall-attached horizontal layers can synchronize as long as their thickness is less than approximately thirty wall units. When the horizontal layer is detached from the wall, the critical thickness increases with height and is proportional to the local wall-normal Taylor microscale. A flow-parallel, vertical layer that spans the height of the channel synchronizes when its spanwise width is of the order of the near-wall Taylor microscale, while the criterion for a crossflow vertical layer is set by the advection distance within a Lyapunov time scale. Finally, we demonstrate that synchronization is possible when only planar velocity data are available, rather than the full outer state, as long as the unknown region satisfies the condition for synchronization in one direction. These numerical experiments demonstrate the capacity of accurately reconstructing, or synchronizing, the missing scales of turbulence from observations, using continuous data assimilation.

Local Resultant Gradient Vector Difference and Inpainting for 3D Text Detection in the Wild

Three-dimensional (3D) text appearing in natural scene images is common due to 3D cameras and the capture of text from different angles, which presents new problems for text detection. This is because of the presence of depth information, shadows, and decorative characters in the images. In this work, we consider those images where 3D text appears with depth, as well as shadow information for text detection. We propose a novel method based on local resultant gradient vector difference (LRGVD), inpainting and a deep learning model for detecting 3D as well as two-dimensional (2D) texts in natural scene images. The boundary of components that are invariant to the above challenges is detected by exploring LRGVD. The LRGVD uses gradient magnitude and direction in a novel way for detecting the boundary of the components. Further, we propose an inpainting method in a new way for restoring the character background information using boundaries. For a given region and the input image, the inpainting method divides the whole image into planes and then propagates the values in the planes into the missing region based on posterior probabilities and neighboring information. This results in text regions with false positives. Then, the differential binarization network (DB-Net) is proposed for detecting text irrespective of orientation, background, 3D or 2D, etc. Experiments conducted on our 3D text images and standard datasets of natural scene text images, namely ICDAR 2019 MLT, ICDAR 2019 ArT, DAST1500, Total-Text and SCUT-CTW1500, show that the proposed method is effective in detecting 3D and 2D texts in the images.

Temporal Group Fusion Network for Deep Video Inpainting

Video inpainting is a task of synthesizing spatio-temporal coherent content in missing regions of the given video sequence, which has recently drawn increasing attention. To utilize the temporal information across frames, most recent deep learning-based methods align reference frames to target frame firstly with explicit or implicit motion estimation and then integrate the information from the aligned frames. However, their performance relies heavily on the accuracy of frame-to-frame alignment. To alleviate the above problem, in this paper, a novel Temporal Group Fusion Network (TGF-Net) is proposed to effectively integrate temporal information through a two-stage fusion strategy. Specifically, the input frames are reorganized into different groups, where each group is followed by an intra-group fusion module to integrate information within the group. Different groups provide complementary information for the missing region. A temporal attention model is further designed to adaptively integrate the information across groups. Such a temporal information fusion way gets rid of the dependence on alignment operations, greatly improving the visual quality and temporal consistency of the inpainted results. In addition, a coarse alignment model is introduced at the beginning of the network to handle videos with large motion. Extensive experiments on DAVIS and Youtube-VOS datasets demonstrate the superiority of our proposed method in terms of PSNR/SSIM values, visual quality and temporal consistency, respectively.

SAGA-net

In this paper, we propose a real-time shape-assisted graph attention neural network to perform local pointcloud repairment. The orderless pointclouds require an effective shape encoder to distill local and global geometric feature descriptors. Previous work has attempted to convert pointcloud representation into a voxelized shape or perform grid-transformations. While these approaches can subsequently allow common convolution operations on the structured data, they either pose additional computational cost or disrupt the local geometric information. We present SAGA-Net, an efficient graph attention neural network framework with a prior shape inquiry protocol that effectively extracts local geometric information, locates the descriptor for the missing region and accurately reconstructs the local region in a real-time manner. Our framework has been validated on a benchmark dataset, ShapeNet. We demonstrate that our framework can repair each partial pointcloud with accuracy surpassing other frameworks in most object categories, and improve the computational efficiency by orders of magnitude in terms of time cost.

Dual feature fusion network: A dual feature fusion network for point cloud completion

Point cloud data in the real world is often affected by occlusion and light reflection, leading to incompleteness of the data. Large-region missing point clouds will cause great deviations in downstream tasks. A dual feature fusion network (DFF-Net) is proposed to improve the accuracy of the completion of a large missing region of the point cloud. First, a dual feature encoder is designed to extract and fuse the global and local features of the input point cloud. Subsequently, a decoder is used to directly generate a point cloud of missing region that retains local details. In order to make the generated point cloud more detailed, a loss function with multiple terms is employed to emphasise the distribution density and visual quality of the generated point cloud. A large number of experiments show that the authors’ DFF-Net is better than the previous state-of-the-art methods in the aspect of point cloud completion.

SIFNet: Free-form image inpainting using color split-inpaint-fuse approach

Recent deep learning-based approaches have shown outstanding performance in generating visually plausible and refined contents for the missing regions in free-form image inpainting tasks. However, most of the existing methods employ a coarse-to-refine approach where the refinement process depends on a single coarse estimation, often leading to texture and structure inconsistencies. Though several existing methods focus on incorporating additional inputs to mitigate this problem, no learning-based studies have investigated the effects of decomposing input corrupted image into luma and chroma images and performing decoupled inpainting of the decomposed components. To this end, we propose a Split-Inpaint-Fuse Network (SIFNet), an end-to-end two-stage inpainting approach that uses a split-inpaint sub-network for separately inpainting the corrupted luma and chroma images using two decoupled branches in the coarse stage and a fusion sub-network for fusing the inpainted luma and chroma images into a refined image in the refinement stage. Additionally, we propose two attention mechanisms for the coarse stage – a progressive context module to find the patch-level feature similarity for the luma image reconstruction and a spatial-channel context module to find important spatial and channel features for the chroma image reconstruction. Experimental results reveal that our Split-Inpaint-Fuse approach outperforms the existing inpainting methods by comparative margins. In addition, extensive ablation studies confirm the effectiveness of the proposed approach, constituting modules and architectural choices.

Error Concealment in the Density Field of a Spatiotemporal Image Sequence.

One of the most difficult challenges of multimedia transmission during the last two decades has been the retrieval of degraded or missing regions of images and videos while maintaining satisfactory perceptual accuracy. The objective is to retrieve lost data by using the similarity between frames. Usually, error concealment (EC) schemes depend on replacing incorrect data with data that are identical to the initial. This is possible because video contains a high degree of self-similarity. This research focuses on applying an EC approach in transform-domain video sequences. To conduct EC on films, they must first be translated to frames and then transformed using one of the available transformations into frequency-domain images. Using successive frames, it is possible to recover lost or incorrect data from images. Intra-coded frames (I-frames) may be used to recreate lost knowledge in predictive (P-frames) and bidirectional predictive frames (B-frames). I-frame knowledge that has been lost may be restored using previous intra-coded frames. The use of wavelet error concealment generated more precise results than the other techniques. In this study, it was discovered that covering faults in the density sector with wavelets produces more reliable results than the other techniques.

Competitive Learning of Facial Fitting and Synthesis Using UV Energy

The three-dimensional morphable model (3DMM) is the most widely used representative model for obtaining a three-dimensional (3-D) face from a target on an image. Although 3DMMs have demonstrated the powerful capability to represent various facial shapes on natural images, they are limited to capturing texture variations of in-the-wild human faces. Based on the fact that fitting a 3-D facial model to an image determines the corresponding UV map, we propose a novel method for facial fitting and synthesis by competitively training two deep learning networks for facial alignment and UV texture completion. When the completion network is trained using well-aligned UV maps, it can model facial textures precisely and, consequently, fill the missing regions more completely. Accordingly, we use a UV completion network, denoted as a UV energy-based generative adversarial network (UV EB-GAN), to discriminate whether a UV map from the alignment network is well aligned by defining the generative loss of the completion network as the energy. Competitive learning facilitates training the completion network without ground-truth facial UV maps and training the alignment network without hard constraints and regularization terms. The proposed network can be trained in an end-to-end manner. The facial texture, albedo, lighting parameters, and 3-D facial shape can be obtained through this network. The results of the experiments on 2-D alignment, 3-D reconstruction, texture synthesis, and illumination estimation verified that the proposed method achieves remarkable improvements over the state-of-the-art methods.

Parallel adaptive guidance network for image inpainting

Motivated by human behavior, dividing inpainting tasks into structure reconstruction and texture generation helps to simplify restoration process and avoid distorted structures and blurry textures. However, most of tasks are ineffective for dealing with large continuous holes. In this paper, we devise a parallel adaptive guidance network(PAGN), which repairs structures and enriches textures through parallel branches, and several intermediate-level representations in different branches guide each other via the vertical skip connection and the guidance filter, ensuring that each branch only leverages the desirable features of another and outputs high-quality contents. Considering that the larger the missing regions are, less information is available. We promote the joint-contextual attention mechanism(Joint-CAM), which explores the connection between unknown and known patches by measuring their similarity at the same scale and at different scales, to utilize the existing messages fully. Since strong feature representation is essential for generating visually realistic and semantically reasonable contents in the missing regions, we further design attention-based multiscale perceptual res2blcok(AMPR) in the bottleneck that extracts features of various sizes at granular levels and obtains relatively precise object locations. Experiments on the public datasets CelebA-HQ, Places2, and Paris show that our proposed model is superior to state-of-the-art models, especially for filling large holes.

Investigating the Extent of Primer Dropout in SARS-CoV-2 Genome Sequences During the Early Circulation of Delta Variants

The SARS-CoV-2 Delta variant, corresponding to the Pangolin lineage B.1.617.2, was first detected in India in July 2020 and rapidly became dominant worldwide. The ARTIC v3 protocol for SARS-CoV-2 whole-genome sequencing, which relies on a large number of PCR primers, was among the first available early in the pandemic, but may be prone to coverage dropouts that result in incomplete genome sequences. A new set of primers (v4) was designed to circumvent this issue in June 2021. In this study, we investigated whether the sequencing community adopted the new sets of primers, especially in the context of the spread of the Delta lineage, in July 2021. Because information about protocols from individual laboratories is generally difficult to obtain, the aims of the study were to identify whether large under-sequenced regions were present in deposited Delta variant genome sequences (from April to August 2021), to investigate the extent of the coverage dropout among all the currently available Delta sequences in six countries, and to propose simple PCR primer modifications to sequence the missing region, especially for the first circulating Delta variants observed in 2021 in Switzerland. Candidate primers were tested on few clinical samples, highlighting the need to further pursue primer optimization and validation on a larger and diverse set of samples.

Spatially adaptive multi-scale contextual attention for image inpainting

Image inpainting is the task to fill missing regions of an image. Recently, researchers have achieved a great performance by using convolutional neural networks (CNNs) with the conventional patch-matching method. Existing methods compute the attention scores, which are based on the similarity of patches between the known and missing regions. Considering that patches at different spatial positions can convey different levels of detail, we propose a spatially adaptive multi-scale attention score that uses the patches of different scales to compute scores for each pixel at different positions. Through experiments on the Paris Street View and Places datasets, our proposal shows slight improvement compared with some related methods on the quantitative evaluation metrics commonly used in the existing methods. Moreover, we found that these quantitative metrics are not appropriate enough considering the subjective impressions of the generated images. Therefore, we conducted subjective evaluation through user study for comparison, which shows that our proposal has superiority of performance generating much more detailed and subjectively plausible images.