Texture Patches Research Articles

A comprehensive end-to-end computer vision framework for restoration and recognition of low-quality engineering drawings

The digitization of engineering drawings is crucial for efficient reuse, distribution, and archiving. Existing computer vision approaches for digitizing engineering drawings typically assume the input drawings have high quality. However, in reality, engineering drawings are often blurred and distorted due to improper scanning, storage, and transmission, which may jeopardize the effectiveness of existing approaches. This paper focuses on restoring and recognizing low-quality engineering drawings, where an end-to-end framework is proposed to improve the quality of the drawings and identify the graphical symbols on them. The framework uses K-means clustering to classify different engineering drawing patches into simple and complex texture patches based on their gray level co-occurrence matrix statistics. Computer vision operations and a modified Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) model are then used to improve the quality of the two types of patches, respectively. A modified Faster Region-based Convolutional Neural Network (Faster R-CNN) model is used to recognize the quality-enhanced graphical symbols. Additionally, a multi-stage task-driven collaborative learning strategy is proposed to train the modified ESRGAN and Faster R-CNN models to improve the resolution of engineering drawings in the direction that facilitates graphical symbol recognition, rather than human visual perception. A synthetic data generation method is also proposed to construct quality-degraded samples for training the framework. Experiments on real-world electrical diagrams show that the proposed framework achieves an accuracy of 98.98% and a recall of 99.33%, demonstrating its superiority over previous approaches. Moreover, the framework is integrated into a widely-used power system software application to showcase its practicality. The reference codes and data can be found at https://github.com/Lattle-y/AI-recognition-for-lq-ed.git Future work will focus on improving the generalizability of the proposed framework to different quality degradation scenarios and extrapolating the application to different engineering domains.

Unsupervised Deep Learning Approach for Characterizing Fractality in Dried Drop Patterns of Differently Mixed Viscum album Preparations

This paper presents a novel unsupervised deep learning methodology for the analysis of self-assembled structures formed in evaporating droplets. The proposed approach focuses on clustering these structures based on their texture similarity to characterize three different mixing procedures (turbulent, laminar, and diffusion-based) applied to produce Viscum album Quercus 10−3 according to the European Pharmacopoeia guidelines for the production of homeopathic remedies. Texture clustering departs from obtaining a comprehensive texture representation of the full texture patch database using a convolutional neural network. This representation is then dimensionally reduced to facilitate clustering through advanced machine learning techniques. Following this methodology, 13 clusters were found and their degree of fractality determined by means of Local Connected Fractal Dimension histograms, which allowed for characterization of the different production modalities. As a consequence, each image was represented as a vector in R13, enabling classification of mixing procedures via support vectors. As a main result, our study highlights the clear differences between turbulent and laminar mixing procedures based on their fractal characteristics, while also revealing the nuanced nature of the diffusion process, which incorporates aspects from both mixing types. Furthermore, our unsupervised clustering approach offers a scalable and automated solution for analyzing the databases of evaporated droplets.

Controllable Garment Image Synthesis Integrated with Frequency Domain Features

AbstractUsing sketches and textures to synthesize garment images is able to conveniently display the realistic visual effect in the design phase, which greatly increases the efficiency of fashion design. Existing garment image synthesis methods from a sketch and a texture tend to fail in working on complex textures, especially those with periodic patterns. We propose a controllable garment image synthesis framework that takes as inputs an outline sketch and a texture patch and generates garment images with complicated and diverse texture patterns. To improve the performance of global texture expansion, we exploit the frequency domain features in the generative process, which are from a Fast Fourier Transform (FFT) and able to represent the periodic information of the patterns. We also introduce a perceptual loss in the frequency domain to measure the similarity of two texture pattern patches in terms of their intrinsic periodicity and regularity. Comparisons with existing approaches and sufficient ablation studies demonstrate the effectiveness of our method that is capable of synthesizing impressive garment images with diverse texture patterns while guaranteeing proper texture expansion and pattern consistency.

PAINT: Photo-realistic Fashion Design Synthesis

In this article, we investigate a new problem of generating a variety of multi-view fashion designs conditioned on a human pose and texture examples of arbitrary sizes, which can replace the repetitive and low-level design work for fashion designers. To solve this challenging multi-modal image translation problem, we propose a novel Photo-reAlistic fashIon desigN synThesis (PAINT) framework, which decomposes the framework into three manageable stages. In the first stage, we employ a Layout Generative Network (LGN) to transform an input human pose into a series of person semantic layouts. In the second stage, we propose a Texture Synthesis Network (TSN) to synthesize textures on all transformed semantic layouts. Specifically, we design a novel attentive texture transfer mechanism for precisely expanding texture patches to the irregular clothing regions of the target fashion designs. In the third stage, we leverage an Appearance Flow Network (AFN) to generate the fashion design images of other viewpoints from a single-view observation by learning 2D multi-scale appearance flow fields. Experimental results demonstrate that our method is capable of generating diverse photo-realistic multi-view fashion design images with fine-grained appearance details conditioned on the provided multiple inputs. The source code and trained models are available at https://github.com/gxl-groups/PAINT .

Generating High-Fidelity Texture in RGB-D Reconstruction using Patches Density Regularization

We propose a texture-mapping-based online RGB-D reconstruction framework that can generate stable and high-fidelity texture results in near real-time. Different from previous texture generation methods that are prone to output blur results (point blending methods) or require huge computational costs (previous texture mapping methods), our approach achieves high-fidelity and fast texture generation by introducing a new patches density regularization and a series of lightweight optimization methods. The patches density regularization addresses a universal but widely neglected problem: the texture patches fragmentation, which causes discontinuous seams and worsens texture results. By adding this regularization to the optimization process, the fragmentation can be effectively alleviated. To maintain high performance, we design lightweight optimization methods to significantly reduce the computational cost during texture generation. An incremental texture update strategy removes redundant triangular mesh, and a hierarchical-aware seam leveling reduces the number of optimization objects by utilizing light-varying smooth features. Furthermore, the enhanced voxel representation ensures the continuity and stability of the global texture results. Extensive experiments demonstrate that our method achieves a state-of-the-art level, even compared to the mainstream online and offline RGB-D reconstruction frameworks.

Texture Inpainting for Photogrammetric Models

AbstractWe devise a technique designed to remove the texturing artefacts that are typical of 3D models representing real‐world objects, acquired by photogrammetric techniques. Our technique leverages the recent advancements in inpainting of natural colour images, adapting them to the specific context. A neural network, modified and trained for our purposes, replaces the texture areas containing the defects, substituting them with new plausible patches of texels, reconstructed from the surrounding surface texture. We train and apply the network model on locally reparametrized texture patches, so to provide an input that simplifies the learning process, because it avoids any texture seams, unused texture areas, background, depth jumps and so on. We automatically extract appropriate training data from real‐world datasets. We show two applications of the resulting method: one, as a fully automatic tool, addressing all problems that can be detected by analysing the UV‐map of the input model; and another, as an interactive semi‐automatic tool, presented to the user as a 3D ‘fixing’ brush that has the effect of removing artefacts from any zone the users paints on. We demonstrate our method on a variety of real‐world inputs and provide a reference usable implementation.

Grading of HCC Biopsy Images using Nucleus and Texture Features.

Hepatocellular carcinoma (HCC) is one of the most critical health problems in the world. For proper treatment, it is important to identify the grade of cancer morbidity from HCC biopsy image. The diagnostic work is not only time-consuming but also subjective. The same biopsy image may be diagnosed as of different grades by different doctors, due to lack of experience or difference in opinion. In this work, we proposed an automatic grading system with classification accuracy matching to an experienced doctor, to help augment the diagnosis process. First, we proposed a segmentation method to isolate all nucleus-like objects present in a biopsy image. Non-target objects (here the target is a single HCC nucleus) present in the biopsy image are isolated too in the segmentation process. To eliminate such non-target objects, we proposed clustering of segmented images and a novel method to filter out target objects. Next, we proposed a two track neural network, where input consists of 2 different images. It combines a single segmented nucleus and a random cropped texture patch of the biopsy image to which the nucleus belongs. At this classifier output, we grade the single nucleus. Finally, a majority voting method is used to identify the grade of the whole biopsy image. We achieved an accuracy of 99.03% for nucleus image grading and 99.67% accuracy for grading biopsy images.

MIPNet

We present MIPNet, a novel approach for SVBRDF mipmapping which preserves material appearance under varying view distances and lighting conditions. As in classical mipmapping, our method explicitly encodes the multiscale appearance of materials in a SVBRDF mipmap pyramid. To do so, we use a tensor-based representation, coping with gradient-based optimization, for encoding anisotropy which is compatible with existing real-time rendering engines. Instead of relying on a simple texture patch average for each channel independently, we propose a cascaded architecture of multilayer perceptrons to approximate the material appearance using only the fixed material channels. Our neural model learns simple mipmapping filters using a differentiable rendering pipeline based on a rendering loss and is able to transfer signal from normal to anisotropic roughness. As a result, we obtain a drop-in replacement for standard material mipmapping, offering a significant improvement in appearance preservation while still boiling down to a single per-pixel mipmap texture fetch. We report extensive experiments on two distinct BRDF models.

Thyroid Nodule Region Estimation using Auto-Regressive Modelling and Machine Learning

Abstract Ultrasound (US) imaging is used for the diagnosis and also evaluation of thyroid nodules. A Thyroid Imaging Reporting and Data System (TIRADS) is used for the risk stratification of thyroid nodules through US images. The composition of thyroid nodules plays an important role in the risk-stratification process. The percentages of cystic and solid components in a thyroid nodule are one of the features that are can be indicative of the risk of malignancy. In this work, we attempt to classify and estimate solid and cystic regions within nodules. 20x20 texture patches were extracted from solid and cystic regions and converted into signals. These signals are decomposed into low, mid, and high-frequency bands using Continuous Wavelet Transform (CWT). A total of 36 features were extracted from the decomposed signals using Auto- Regressive Modeling. The features were fed into three different Machine Learning (ML) algorithms (Artificial Neural Networks, K-Nearest Neighbors, and Random Forest Classifier) to provide us with a classification of solid versus cystic regions in thyroid nodule US images. The Random Forest Classifier obtained an Accuracy, Sensitivity, and Specificity of 90.41%, 99% and 91% respectively which was the highest among the three chosen ML algorithms. Additionally, the output from the classification phase was also be used to determine the percentage of cystic and solid regions with a given thyroid nodule US image.

Photo-realistic image synthesis from lines and appearance with modular modulation

The image-to-image translation task has made significant progress by relying on conditional generative adversarial networks. However, for many tasks, multiple condition images are required. This paper considers a very classic application scenario, using lines and appearance to synthesize photo-realistic images, describing structure and appearance information, respectively, for example, generating realistic face images from portrait drawings and color scribbles, and generating photos from sketches and texture patches. The key to this type of task is how to fuse the two conditional information. We propose an image translation system driven by line and appearance images, introducing a modular architecture for condition fusion. Unlike the previous condition fusion schemes, its main body of the generator is composed of stacked modulation units (MUs). Here, structural features and appearance features are progressively incorporated via cascaded MUs, each of which pays attention to the local regions. The visualization experiment shows that such a scheme lets the network automatically learn to decompose the fusion process as multiple sub-steps in latent spaces. Our model produces higher quality results quantitatively and qualitatively compared to the state-of-the-art method on different tasks and datasets. The ablation study demonstrates the effectiveness of the MUs and intuitively explains the process of feature fusion through visualization.

Perceptual Aliasing++: Adversarial Attack for Visual SLAM Front-End and Back-End

Similar appearing places are known to confuse place recognition methods (a.k.a perceptual aliasing). Robust visual SLAM methods have shown impressive resilience against scenarios having moderate perceptual aliasing. In this work, we evaluate visual SLAM in scenarios where perceptual aliasing occurs with greater frequency. Firstly we evaluate visual SLAM front-ends. We modify the environment by replicating a simple high textured patch. Surprisingly, this simple patch manages to by-pass all the geometric checks in the state of the art ORB-SLAM, resulting in false loop closures and hence corrupting the map. The patch can be printed on a simple paper, using a simple printer, or it can be digital content displayed on screens. We show that closeups (approaching doors, turning a corridor, interacting with an object) are the most vulnerable situations. This vulnerability exists in multiple loop closing pipelines. Secondly, we evaluate robust SLAM back-ends. These back-ends have shown the ability to recover from false positives under multiple policies (random, local, random grouped and locally grouped). We propose a novel policy, i.e. locally symmetric, for generating false loop closures which successfully attacks multiple robust back-ends and public datasets. Surprisingly, for commonly occurring weaker structures (multiple floors) even a single false loop manages to fool multiple robust back-ends. We hope these findings will motivate the community in evolving SLAM solutions for adversarial environments. To further enhance progress in this direction, we release a novel dataset with adversarial content in both visual SLAM front-ends and back-ends <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

TextureMe: High-Quality Textured Scene Reconstruction in Real Time

Three-dimensional (3D) reconstruction using an RGB-D camera has been widely adopted for realistic content creation. However, high-quality texture mapping onto the reconstructed geometry is often treated as an offline step that should run after geometric reconstruction. In this article, we propose TextureMe , a novel approach that jointly recovers 3D surface geometry and high-quality texture in real time. The key idea is to create triangular texture patches that correspond to zero-crossing triangles of truncated signed distance function (TSDF) progressively in a global texture atlas. Our approach integrates color details into the texture patches in parallel with the depth map integration to a TSDF. It also actively updates a pool of texture patches to adapt TSDF changes and minimizes misalignment artifacts that occur due to camera drift and image distortion. Our global texture atlas representation is fully compatible with conventional texture mapping. As a result, our approach produces high-quality textures without utilizing additional texture map optimization, mesh parameterization, or heavy post-processing. High-quality scenes produced by our real-time approach are even comparable to the results from state-of-the-art methods that run offline.

Perceptual Texture Dimensions Modulate Neuronal Response Dynamics in Visual Cortical Area V4.

Texture is an important visual attribute for surface pattern discrimination and therefore object segmentation, but the neural bases of texture perception are largely unknown. Previously, we demonstrated that the responses of V4 neurons to naturalistic texture patches are sensitive to four key features of human texture perception: coarseness, directionality, regularity, and contrast. To begin to understand how distinct texture perception emerges from the dynamics of neuronal responses, in 2 macaque monkeys (1 male, 1 female), we investigated the relative contribution of the four texture attributes to V4 responses in terms of the strength and timing of response modulation. We found that the different feature dimensions are associated with different temporal dynamics. Specifically, the response modulation associated with directionality and regularity was significantly delayed relative to that associated with coarseness and contrast, suggesting that the latter are fundamentally simpler feature dimensions. The population of texture-selective neurons could be grouped into multiple clusters based on the combination of feature dimensions encoded, and those subpopulations displayed distinct temporal dynamics characterized by the weighted combinations of multiple features. Finally, we applied a population decoding approach to demonstrate that texture category information can be obtained from short temporal windows across time. These results demonstrate that the representation of different perceptually relevant texture features emerge over time in the responses of V4 neurons. The observed temporal organization provides a framework to interpret how the processing of surface features unfolds in early and midlevel cortical stages, and could ultimately inform the interpretation of perceptual texture dynamics.SIGNIFICANCE STATEMENT To delineate how neuronal responses underlie our ability to perceive visual textures, we related four key perceptual dimensions (coarseness, directionality, regularity, and contrast) of naturalistic textures to the strength and timing of modulation of neuronal responses in area V4, an intermediate stage in the form-processing, ventral visual pathway. Our results provide the first characterization of V4 temporal dynamics for texture encoding along perceptually defined axes.

Machine learning based texture analysis of patella from X-rays for detecting patellofemoral osteoarthritis

ObjectiveTo assess the ability of texture features for detecting radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. DesignWe used lateral view knee radiographs from The Multicenter Osteoarthritis Study (MOST) public use datasets (n = 5507 knees). Patellar region-of-interest (ROI) was automatically detected using landmark detection tool (BoneFinder), and subsequently, these anatomical landmarks were used to extract three different texture ROIs. Hand-crafted features, based on Local Binary Patterns (LBP), were then extracted to describe the patellar texture. First, a machine learning model (Gradient Boosting Machine) was trained to detect radiographic PFOA from the LBP features. Furthermore, we used end-to-end trained deep convolutional neural networks (CNNs) directly on the texture patches for detecting the PFOA. The proposed classification models were eventually compared with more conventional reference models that use clinical assessments and participant characteristics such as age, sex, body mass index (BMI), the total Western Ontario and McMaster Universities Arthritis Index (WOMAC) score, and tibiofemoral Kellgren-Lawrence (KL) grade. Atlas-guided visual assessment of PFOA status by expert readers provided in the MOST public use datasets was used as a classification outcome for the models. Performance of prediction models was assessed using the area under the receiver operating characteristic curve (ROC AUC), the area under the precision-recall (PR) curve -average precision (AP)-, and Brier score in the stratified 5-fold cross validation setting. ResultsOf the 5507 knees, 953 (17.3%) had PFOA. AUC and AP for the strongest reference model including age, sex, BMI, WOMAC score, and tibiofemoral KL grade to predict PFOA were 0.817 and 0.487, respectively. Textural ROI classification using CNN significantly improved the prediction performance (ROC AUC = 0.889, AP = 0.714). ConclusionWe present the first study that analyses patellar bone texture for diagnosing PFOA. Our results demonstrates the potential of using texture features of patella to predict PFOA.

Reducing reconstruction error of classified textural patches by integration of random forests and coupled dictionary nonlinear regressors: with applications to super-resolution of abdominal CT images.

Random forests and dictionary-based statistical regressions have common characteristics, including non-linear mapping and supervised learning. To reduce the reconstruction error of high-resolution images, we integrate random forests and coupled dictionary learning. Textural differences of image blocks are considered by the classification of patches using an Auto-Encoder network. The proposed algorithm partitions an input LR image by 5 × 5 blocks and classifies training patches into six categories. A single random forest regressor is then trained corresponding to each class. The output of an RF is considered as an initial estimate of the HR slice. If a slice's representation is sparse in the Discrete Cosine Transform domain, the initial reconstructed image is further improved by a coupled dictionary. In this study, we applied our method to abdominal CT scans and compared them to conventional and recent researches. We achieved an average improvement of 0.06 (2.37) using the SSIM (PSNR) index compared to the random forest + dictionary learning method. The low standard deviation of the results reveals the stability of the proposed method as well. The proposed algorithm depicts the effectiveness of classifying image patches and individual treatment of each class.

Image Scale Estimation Using Surface Textures for Quantitative Visual Inspection

In this study, a learning-based scale estimation technique is proposed to enable quantitative evaluation of inspection regions. The underlying idea is that surface texture of structures (i.e. bridges or buildings) captured on images contains the scale information of the corresponding images, which is represented by pixel per physical dimension (e.g., mm, inch). This allows training a regression model that provides a relationship between surface textures on images and their corresponding scales. Deep convolutional neural network is used to extract scale-related features from the texture patches and estimate their scales. The trained model can be exploited to estimate scales for all images captured from structure surfaces that have similar textures. The capability of the proposed technique is fully demonstrated using data collected from surface textures of three different structures and achieves an overall average scale estimation error of less than 15%.

Image restoration based on sparse representation using feature classification learning

In the image inpainting method based on sparse representation, the adaptability of over-complete dictionary has a great influence on the result of image restoration. If the over-complete dictionary cannot effectively reflect the differences between different local features, it may result in the loss of texture details, resulting in blurred or over-smooth phenomenon in restored images. In view of these problems, we propose an image restoration method based on sparse representation using feature classification learning. Firstly, we perform singular value decomposition on the local gradient vector. According to the relationship between the main orientation and the secondary orientation, we classify all the local patches into three categories: smooth patch, edge patch and texture patch. Secondly, we use K-Singular Value Decomposition method to learn over-complete dictionaries that adapt to different features. Finally, we use Orthogonal Matching Pursuit method to calculate the sparse coding of target patches with different local features on their corresponding over-complete dictionaries, and use the over-complete dictionary and corresponding sparse coding to restore the damaged pixels. A series of experiments on various restoration tasks show the superior performance of the proposed method.

Real-Time Globally Consistent Dense 3D Reconstruction With Online Texturing.

High-quality reconstruction of 3D geometry and texture plays a vital role in providing immersive perception of the real world. Additionally, online computation enables the practical usage of 3D reconstruction for interaction. We present an RGBD-based globally-consistent dense 3D reconstruction approach, where high-quality (i.e., the spatial resolution of the RGB image) texture patches are mapped on high-resolution ([Formula: see text]) geometric models online. The whole pipeline uses merely the CPU computing of a portable device. For real-time geometric reconstruction with online texturing, we propose to solve the texture optimization problem with a simplified incremental MRF solver in the context of geometric reconstruction pipeline using sparse voxel sampling strategy. An efficient reference-based color adjustment scheme is also proposed to achieve consistent texture patch colors under inconsistent luminance situations. Quantitative and qualitative experiments demonstrate that our online scheme achieves a realistic visualization of the environment with more abundant details, while taking fairly compact memory consumption and much lower computational complexity than existing solutions.

The Characterization of Local Clay from Anambra State Nigeria for Its Potential Applications in the Production of Ceramic Whitewares

Local clay deposits from Ukpor in Nnewi South Local Government Area of Anambra State were characterized for its potential applications in the production of ceramics whitewares. XRF analysis of the clay revealed that it has high silica and alumina contents with some flux oxides. XRD analysis also revealed that of all the minerals present in the clay, quartz has the highest constituent followed by kaolinite and rutile. FT-IR analysis revealed that the bands corresponds to 3500 cm-1 to 4000 cm-1 indicating the presence hydroxyl groups (OH) and Si-O symmetry vibration stretch at 1206.207 cm-1 which confirmed the kaolinite structure of the clay. The SEM study showed a finely distributed uniform surface with isolated patches of granulated texture which explained the physical behaviour of the clay. The EDS spectrum of the formulated fired Ukpor clay indicated the presence of O, Al, K, Ca, K, Fe, and Si. The study revealed that an increase in the firing temperature and feldspar contents of the clay increases its bulk density, while the apparent density decreases with increase in temperature. Apparent porosity showed that there was a decrease from 25% to 3%, and the water absorption increases with increase in the silica content and decreases with temperature and feldspar increase. The shrinkage increases with the increase in temperature. The results concluded that the clay sample could be utilized as an industrial raw materials for the production of ceramic whitewares.

Self‐guided filter for image denoising

The guided filter has been acknowledged as an exceptional edge-preserving filter whose output is a locally linear transform of the guidance image. However, the traditional guided filter heavily relies on the guidance image and fails to achieve the desired result when performing image denoising without a clear guidance image. In this study, to address this limitation, the authors propose a simple yet effective guided filter variant for the single image noise removing. They further show that the proposed denoising strategy can be easily realised by using the iterative framework. Moreover, the weak textured patches based image noise estimation is utilised to generate a clear intermediate image which makes the proposed method highly adaptable to the local noise level. Experimental results demonstrate that their proposed algorithm can compete with the state-of-the-art local denoising methods in edge-preserving.