High-resolution Texture Research Articles

Lightweight Reference-Based Video Super-Resolution Using Deformable Convolution

Super-resolution is a technique for generating a high-resolution image or video from a low-resolution counterpart by predicting natural and realistic texture information. It has various applications such as medical image analysis, surveillance, remote sensing, etc. However, traditional single-image super-resolution methods can lead to a blurry visual effect. Reference-based super-resolution methods have been proposed to recover detailed information accurately. In reference-based methods, a high-resolution image is also used as a reference in addition to the low-resolution input image. Reference-based methods aim at transferring high-resolution textures from the reference image to produce visually pleasing results. However, it requires texture alignment between low-resolution and reference images, which generally requires a lot of time and memory. This paper proposes a lightweight reference-based video super-resolution method using deformable convolution. The proposed method makes the reference-based super-resolution a technology that can be easily used even in environments with limited computational resources. To verify the effectiveness of the proposed method, we conducted experiments to compare the proposed method with baseline methods in two aspects: runtime and memory usage, in addition to accuracy. The experimental results showed that the proposed method restored a high-quality super-resolved image from a very low-resolution level in 0.0138 s using two NVIDIA RTX 2080 GPUs, much faster than the representative method.

GF-4 high-resolution texture and FY-4A multispectral data fusion: Two case studies for enhancing early convective cloud detection

Early detection of convective clouds is vital for minimizing hazardous impacts. Forecasting convective initiation (CI) using current multispectral geostationary meteorological satellites is often challenged by high false-alarm rates and missed detections caused by limited resolution. In contrast, high-resolution earth observation satellites offer more detailed texture information, improving early detection capabilities. The authors propose a novel methodology that integrates the advanced features of China's latest-generation satellites, Gaofen-4 (GF-4) and Fengyun-4A (FY-4A). This fusion method retains GF's high-resolution details and FY-4A's multispectral information. Two cases from different observational scenarios and weather conditions under GF-4's staring mode were carried out to compare the CI forecast results based on fused data and solely on FY-4A data. The fused data demonstrated superior performance in detecting smaller-scale convective clouds, enabling earlier forecasting with a lead time of 15–30 minutes, and more accurate location identification. Integrating high-resolution earth observation satellites into early convective cloud detection provides valuable insights for forecasters and decision-makers, particularly given the current resolution limitations of geostationary meteorological satellites.摘要对流云的早期探测对于降低天气致灾风险至关重要. 当前静止气象卫星受其分辨率的限制, 在预测对流初生 (CI) 时仍存在较高的虚警率和漏报率. 对比而言, 高分辨率的地球观测卫星能够提供更丰富的纹理细节, 有助于提升对流云早期探测的能力. 作者提出了一种创新性的方法, 融合GF-4的高分辨率细节和FY-4A的多光谱信息, 以优化对流云的早期识别和预测. 通过对GF-4凝视模式下两种不同天气条件和观测场景的案例分析, 融合数据在探测更小尺度的对流云方面具有明显优势, 一定程度解决漏报难题, 另一方面能够将对流初生的可探测时间提前, 更精确表现对流爆发位置和更准确刻画对流发展过程. 在当前静止气象卫星分辨率受限的条件下, 这种高分辨率地球观测卫星数据的融合应用能够为气象预报员和决策者提供更有价值的观测视角.

Application research of interior space design elements based on image processing

In order to quantitatively analyze the application performance of elements in interior space design, a detection method of texture visual features of elements distribution images in interior space design based on image processing and texture edge contour features detection is proposed. A multi-resolution visual information collection model for detecting the texture visual characteristics of element distribution images in indoor space design is constructed, and a visual characteristic identification model for element application configuration in indoor space design is established. Multi-texture grid block area combination design method is adopted to realize the fusion of element application configuration visual images in indoor space design. Through the edge feature detection method of texture grid area, Establish a visual parameter identification model of element application configuration in interior space design, extract the visual information feature quantity of element application configuration in interior space design, adopt Harris corner feature matching method, realize distributed detection of texture edge contour features of element application configuration visual features in interior space design, and fuse detection of high-resolution texture information through combined filtering and collaborative filtering. The collection and optimization of visual features of element application configuration in interior space design are realized, and the distributed detection algorithm of texture edge contour features is adopted to improve the convergence level of detection of texture visual features of element distribution images in interior space design, and the application of elements in interior space design based on image processing is realized. The simulation results show that this method has good resolution and strong visual expression ability in extracting the texture visual features of element distribution images in interior space design, which improves the quantitative analysis ability of elements in interior space design.

OpenSVBRDF: A Database of Measured Spatially-Varying Reflectance

We present the first large-scale database of measured spatially-varying anisotropic reflectance, consisting of 1,000 high-quality near-planar SVBRDFs, spanning 9 material categories such as wood, fabric and metal. Each sample is captured in 15 minutes, and represented as a set of high-resolution texture maps that correspond to spatially-varying BRDF parameters and local frames. To build this database, we develop a novel integrated system for robust, high-quality and -efficiency reflectance acquisition and reconstruction. Our setup consists of 2 cameras and 16,384 LEDs. We train 64 lighting patterns for efficient acquisition, in conjunction with a network that predicts per-point reflectance in a neural representation from carefully aligned two-view measurements captured under the patterns. The intermediate results are further fine-tuned with respect to the photographs acquired under 63 effective linear lights, and finally fitted to a BRDF model. We report various statistics of the database, and demonstrate its value in the applications of material generation, classification as well as sampling. All related data, including future additions to the database, can be downloaded from https://opensvbrdf.github.io/.

High-Resolution Iterative Feedback Network for Camouflaged Object Detection

Spotting camouflaged objects that are visually assimilated into the background is tricky for both object detection algorithms and humans who are usually confused or cheated by the perfectly intrinsic similarities between the foreground objects and the background surroundings. To tackle this challenge, we aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries. We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner, essentially a global loop-based connection among the multi-scale resolutions. To design better feedback feature flow and avoid the feature corruption caused by recurrent path, an iterative feedback strategy is proposed to impose more constraints on each feedback connection. Extensive experiments on four challenging datasets demonstrate that our HitNet breaks the performance bottleneck and achieves significant improvements compared with 29 state-of-the-art methods. In addition, to address the data scarcity in camouflaged scenarios, we provide an application example to convert the salient objects to camouflaged objects, thereby generating more camouflaged training samples from the diverse salient object datasets. Code will be made publicly available.

Perceptual Substitution Based Haptic Texture Rendering for Narrow-Band Reproduction.

Recorded high-resolution texture vibration contains perceptually redundant spectral information due to tactile limitations of human skin. Also, accurate reproduction of recorded texture vibration is often infeasible for widely available haptic reproduction systems at mobile devices. Usually, haptic actuators can only reproduce narrow-bandwidth vibration. With the exception of research setups, rendering strategies need to be developed, that utilize the limited capabilities of various actuator systems and tactile receptors while minimizing a negative impact on perceived quality of reproduction. Therefore, the aim of this study is to substitute recorded texture vibrations with perceptually sufficient simple vibrations. Accordingly, similarity of band-limited noise, single sinusoid and amplitude-modulated signals on display are rated compared to real textures. Considering that low and high frequency bands of noise signals might be implausible and redundant, different combinations of cut-off frequencies are applied to noise vibrations. Moreover, suitability of amplitude-modulation signals are tested for coarse textures in addition to single sinusoids because of their capability of creating pulse-like roughness sensation without too low frequencies. With the set of experiments, narrowest band noise vibration with frequencies between 90 Hz to 400 Hz is determined according to the fine textures. Furthermore, AM vibrations are found to be more congruent than single sinusoids to reproduce too coarse textures.

Mesh-Based DGCNN: Semantic Segmentation of Textured 3-D Urban Scenes

Textured 3D mesh is one of the final user products in photogrammetry and remote sensing. However, research on the semantic segmentation of complex urban scenes represented by textured 3D meshes is in its infancy. We present a mesh-based dynamic graph CNN (DGCNN) for the semantic segmentation of textured 3D meshes. To represent each mesh facet, composite input feature vectors are constructed by concatenating the face-inherent features, i.e., XYZ coordinates of the center of gravity (CoG), texture values, and normal vectors. A texture fusion module is embedded into the proposed mesh-based DGCNN to generate high-level semantic features of the high-resolution texture information, which is useful for semantic segmentation. We achieve competitive accuracies when the proposed method is applied to the SUM mesh datasets. The overall accuracy (OA), Kappa coefficient (Kap), mean precision (mP), mean recall (mR), mean F1 score (mF1), and mean intersection over union (mIoU) are 93.3%, 88.7%, 79.6%, 83.0%, 80.7%, and 69.6%, respectively. In particular, the OA, mean class accuracy (mAcc), mIoU, and mF1 increase by 0.3%, 12.4%, 3.4%, and 6.9%, respectively, compared to the state-of-the-art method.

Rapid Face Asset Acquisition with Recurrent Feature Alignment

We present Re current F eature A lignment (ReFA), an end-to-end neural network for the very rapid creation of production-grade face assets from multi-view images. ReFA is on par with the industrial pipelines in quality for producing accurate, complete, registered, and textured assets directly applicable to physically-based rendering, but produces the asset end-to-end, fully automatically at a significantly faster speed at 4.5 FPS, which is unprecedented among neural-based techniques. Our method represents face geometry as a position map in the UV space. The network first extracts per-pixel features in both the multi-view image space and the UV space. A recurrent module then iteratively optimizes the geometry by projecting the image-space features to the UV space and comparing them with a reference UV-space feature. The optimized geometry then provides pixel-aligned signals for the inference of high-resolution textures. Experiments have validated that ReFA achieves a median error of 0.603 mm in geometry reconstruction, is robust to extreme pose and expression, and excels in sparse-view settings. We believe that the progress achieved by our network enables lightweight, fast face assets acquisition that significantly boosts the downstream applications, such as avatar creation and facial performance capture. It will also enable massive database capturing for deep learning purposes.

Visualizing Dunhuang: The Lo Archive Photographs of the Mogao and Yulin Caves

Visualizing Dunhuang: The Lo Archive Photographs of the Mogao and Yulin Caves

A Novel Deep-Learning-Based Enhanced Texture Transformer Network for Reference Image Super-Resolution

The study explored a deep learning image super-resolution approach which is commonly used in face recognition, video perception and other fields. These generative adversarial networks usually have high-frequency texture details. The relevant textures of high-resolution images could be transferred as reference images to low-resolution images. The latest existing methods use transformer ideas to transfer related textures to low-resolution images, but there are still some problems with channel learning and detailed textures. Therefore, the study proposed an enhanced texture transformer network (ETTN) to improve the channel learning ability and details of the texture. It could learn the corresponding structural information of high-resolution texture images and convert it into low-resolution texture images. Through this, finding the feature map can change the exact feature of images and improve the learning ability between channels. We then used multi-scale feature integration (MSFI) to further enhance the effect of fusion and achieved different degrees of texture restoration. The experimental results show that the model has a good resolution enhancement effect on texture transformers. In different datasets, the peak signal to noise ratio (PSNR) and structural similarity (SSIM) were improved by 0.1–0.5 dB and 0.02, respectively.

Image Super-Resolution Reconstruction Method for Lung Cancer CT-Scanned Images Based on Neural Network.

The super-resolution (SR) reconstruction of a single image is an important image synthesis task especially for medical applications. This paper is studying the application of image segmentation for lung cancer images. This research work is utilizing the power of deep learning for resolution reconstruction for lung cancer-based images. At present, the neural networks utilized for image segmentation and classification are suffering from the loss of information where information passes through one layer to another deep layer. The commonly used loss functions include content-based reconstruction loss and generative confrontation network. The sparse coding single-image super-resolution reconstruction algorithm can easily lead to the phenomenon of incorrect geometric structure in the reconstructed image. In order to solve the problem of excessive smoothness and blurring of the reconstructed image edges caused by the introduction of this self-similarity constraint, a two-layer reconstruction framework based on a smooth layer and a texture layer is proposed for a medical application of lung cancer. This method uses a global nonzero gradient number constrained reconstruction model to reconstruct the smooth layer. The proposed sparse coding method is used to reconstruct high-resolution texture images. Finally, a global and local optimization models are used to further improve the quality of the reconstructed image. An adaptive multiscale remote sensing image super-division reconstruction network is designed. The selective core network and adaptive gating unit are integrated to extract and fuse features to obtain a preliminary reconstruction. Through the proposed dual-drive module, the feature prior drive loss and task drive loss are transmitted to the super-resolution network. The proposed work not only improves the subjective visual effect but the robustness has also been enhanced with more accurate construction of edges. The statistical evaluators are used to test the viability of the proposed scheme.

Classification of Flexible Pavements Based on Texture Data

The strong correlations between pavement texture and other surface characteristics—such as friction, drainage, and noise—are widely known. However, those relationships are not straightforward and have a strong dependency on the type and properties of the pavement surface. Incorporating surface information into a prediction model for skid resistance has the potential to drastically increase its predictive power. Nonetheless, in practice, pavement surface information is often unknown at the network level and must be inferred based on local expertise. The objective of this study was to address this issue by developing an objective classification model capable of identifying different asphalt pavement surfaces with a high degree of accuracy using only field texture data. High-resolution texture data were collected on 21 highway sections with varying surface types within 60 mi of the city of Austin, Texas using a prototype developed in house. Sophisticated data-processing algorithms were created to ensure the data used in developing the classification model were of the highest possible quality. Based on the most recent literature on the topic, more than 20 texture summary statistics were assessed during this study to find the best combination to predict surface characteristics of asphalt pavements. The results are robust and indicate that texture summary statistics alone, when used in the right combination, have enough information to develop a pavement surface classification model with a predictive power as high as 89% based on the F1 score.

A Workflow for Digitizing Anatomical Specimens by Combining Photogrammetry and 3D Scanning

The 3D visualization of anatomical specimens relies on two main technologies: 3D laser surface scanning and photogrammetry (PGM). While 3D laser scanning provides the most accurate 3D model, photogrammetry provides the best, photorealistic surface texture. Typically, a specimen would be assessed to see which technology is best to capture it ‐ in this study we set out to investigate whether it is feasible to combine both methods, incorporating the advantages of both. Artec Space Spider scanner was used to capture accurate geometry of specimens and create a mesh with high polygon count and topological accuracy in Artec Studio 15, and Reality Capture was used to capture photographs around the specimen at regular 5 degree increments until the entire surface was captured to produce a high‐resolution textured mesh. The meshes were combined in Artec Studio 15 by matching their geometric centers in 3D space and ensuring that all polygons are precisely overlapped ‐ even a minor amount of deviation would result in blurry reprojection of texture. This is followed by mapping and reprojecting the PGM texture on the 3D scanned mesh (now aligned and scaled) in Reality Capture and cleaning up the produced texture map in Adobe Photoshop to ensure that any artifacts are cleaned up. Through this combination, models that gather ‘best of both worlds’ qualities from 3D scanning and photogrammetry ‐ laser accurate geometry and photorealism. We determined that 3D scanning is sufficient for specimens with simple texture, and photogrammetry is effective for specimens with complex texture and simple geometry; the combination method shines where great detail and high resolution texture is required to meet the learning objectives, such as organs and musculoskeletal specimens and other specimens with key topological details and colorations. Gathering both PGM and 3D scanning data allows for error correction, especially when performed in collaboration with a content expert and a 3D artist. Availability of these methods to produce photorealistic and accurate scans of specimens lends to accessibility of high quality online anatomy educational materials for students. This technology may also be useful in museum settings and to scan rare collections where to date a single method was not able to capture the complexity of the specimens.

Improved TLBO for Fusion of Infrared and Visible Images

Image fusion is an image enhancement method in modern artificial intelligence theory, which can reduce the pressure in data storage and obtain better image information. Due to different imaging principles, information of the infrared image and visible images’ information is complementary and redundant. The infrared image can be fused with a visible image to obtain both the high-resolution texture details and the edge contour of the infrared image. In this paper, the fusion algorithm of forest sample image is studied at the feature level, which aims to accurately extract tree features through information fusion, ensure data stability and reliability, and improve the accuracy of target recognition. The main research contents of this paper are as follows: (1) teaching learning-based optimization (TLBO) algorithm was used to optimize the weighted coefficient in the fusion process, and the value range of random parameters in the model was adjusted to optimize the fusion effect. Compared with before optimization, image information increased by 2.05%, and spatial activity increased by 15.27%. (2) Experimental data show that the target recognition accuracy of feature-level fusion results was 93.6%, 13.9% higher than that of the original infrared sample image, and 18.8% higher than that of the original visible sample image. Pixel-level and feature-level fusion have their characteristics and application scopes. This method can improve the quality of the specified region in the image and is suitable for detecting intelligent information in forest regions.

High-Resolution Neural Texture Synthesis with Long-Range Constraints

The field of texture synthesis has witnessed important progresses over the last years, most notably through the use of convolutional neural networks. However, neural synthesis methods still struggle to reproduce large-scale structures, especially with high-resolution textures. To address this issue, we first introduce a simple multi-resolution framework that efficiently accounts for long-range dependency. Then, we show that additional statistical constraints further improve the reproduction of textures with strong regularity. This can be achieved by constraining both the Gram matrices of a neural network and the power spectrum of the image. Alternatively, one may constrain only the autocorrelation of the features of the network and drop the Gram matrices constraints. In an experimental part, the proposed methods are then extensively tested and compared to alternative approaches, both in an unsupervised way and through a user study. Experiments show the advantage of the multi-scale scheme for high-resolution textures and the advantage of combining it with additional constraints for regular textures.

HRTex: a high-resolution texture data processing tool for monochromatic neutron diffraction based on the pixel projection method

HRTex is a new texture data processing tool for two-dimensional position-sensitive area detectors on monochromatic neutron diffractometers. With the aim of improving the resolution and accuracy of pole figure calculations, HRTex treats the raw data of the area detector for each pixel and projects the intensity of each pixel directly onto a high-resolution pole figure. With the resultant refinement of the resolution, HRTex can distinguish close texture peaks with a flexible resolution setting and reduced information loss. Test results of HRTex on the data sets of two samples measured by two different neutron facilities are analysed, and the improvements in accuracy, resolution and efficiency of the pole figure calculation are discussed.

Multimode fusion perception for transparent glass recognition

PurposeThe purpose of this paper is to present a novel tactile sensor and a visual-tactile recognition framework to reduce the uncertainty of the visual recognition of transparent objects.Design/methodology/approachA multitask learning model is used to recognize intuitive appearance attributes except texture in the visual mode. Tactile mode adopts a novel vision-based tactile sensor via the level-regional feature extraction network (LRFE-Net) recognition framework to acquire high-resolution texture information and temperature information. Finally, the attribute results of the two modes are integrated based on integration rules.FindingsThe recognition accuracy of attributes, such as style, handle, transparency and temperature, is near 100%, and the texture recognition accuracy is 98.75%. The experimental results demonstrate that the proposed framework with a vision-based tactile sensor can improve attribute recognition.Originality/valueTransparency and visual differences make the texture of transparent glass hard to recognize. Vision-based tactile sensors can improve the texture recognition effect and acquire additional attributes. Integrating visual and tactile information is beneficial to acquiring complete attribute features.

Textured Mesh Generation Using Multi-View and Multi-Source Supervision and Generative Adversarial Networks

This study focuses on reconstructing accurate meshes with high-resolution textures from single images. The reconstruction process involves two networks: a mesh-reconstruction network and a texture-reconstruction network. The mesh-reconstruction network estimates a deformation map, which is used to deform a template mesh to the shape of the target object in the input image, and a low-resolution texture. We propose reconstructing a mesh with a high-resolution texture by enhancing the low-resolution texture through use of the super-resolution method. The architecture of the texture-reconstruction network is like that of a generative adversarial network comprising a generator and a discriminator. During the training of the texture-reconstruction network, the discriminator must focus on learning high-quality texture predictions and to ignore the difference between the generated mesh and the actual mesh. To achieve this objective, we used meshes reconstructed using the mesh-reconstruction network and textures generated through inverse rendering to generate pseudo-ground-truth images. We conducted experiments using the 3D-Future dataset, and the results prove that our proposed approach can be used to generate improved three-dimensional (3D) textured meshes compared to existing methods, both quantitatively and qualitatively. Additionally, through our proposed approach, the texture of the output image is significantly improved.

3D-FUTURE: 3D Furniture Shape with TextURE

The 3D CAD shapes in current 3D benchmarks are mostly collected from online model repositories. Thus, they typically have insufficient geometric details and less informative textures, making them less attractive for comprehensive and subtle research in areas such as high-quality 3D mesh and texture recovery. This paper presents 3D Furniture shape with TextURE (3D-FUTURE): a richly-annotated and large-scale repository of 3D furniture shapes in the household scenario. At the time of this technical report, 3D-FUTURE contains 9992 modern 3D furniture shapes with high-resolution textures and detailed attributes. To support the studies of 3D modeling from images, we couple the CAD models with 20,240 scene images. The room scenes are designed by professional designers or generated by an industrial scene creating system. Given the well-organized 3D-FUTURE and its characteristics, we provide a package of baseline experiments, such as joint 2D instance segmentation and 3D object pose estimation, image-based 3D shape retrieval, 3D object reconstruction from a single image, texture recovery for 3D shapes, and furniture composition, to facilitate related future researches on our database.

Three-dimensional shape reconstruction of uncooperative spacecraft with texture-guided depth upsampling

An important aspect of space exploration and situational awareness involves the characterization of the surface geometry of space objects. This paper puts forward a precise three-dimensional (3D) reconstruction method for uncooperative spacecraft based on a low-resolution time-of-flight (ToF) depth camera coupled with a high-resolution optical texture camera. In the proposed approach, a texture-guided depth upsampling algorithm is adopted to reconstruct high-resolution 3D shapes with high-precision geometry and detailed textures, in which the low-resolution depth map upsampling is guided by the high-resolution texture image. Then, in order to build a globally consistent 3D surface model, a scale estimator-based iterative closest point algorithm is put forward to align the 3D shapes of the uncooperative spacecraft from different views. The experimental results demonstrate both quantitatively and qualitatively that our approach achieves a significant improvement in reconstruction results compared to ToF-based solutions.