Image Texture Research Articles

ConvMambaSR: Leveraging State-Space Models and CNNs in a Dual-Branch Architecture for Remote Sensing Imagery Super-Resolution

Deep learning-based super-resolution (SR) techniques play a crucial role in enhancing the spatial resolution of images. However, remote sensing images present substantial challenges due to their diverse features, complex structures, and significant size variations in ground objects. Moreover, recovering lost details from low-resolution remote sensing images with complex and unknown degradations, such as downsampling, noise, and compression, remains a critical issue. To address these challenges, we propose ConvMambaSR, a novel super-resolution framework that integrates state-space models (SSMs) and Convolutional Neural Networks (CNNs). This framework is specifically designed to handle heterogeneous and complex ground features, as well as unknown degradations in remote sensing imagery. ConvMambaSR leverages SSMs to model global dependencies, activating more pixels in the super-resolution task. Concurrently, it employs CNNs to extract local detail features, enhancing the model’s ability to capture image textures and edges. Furthermore, we have developed a global–detail reconstruction module (GDRM) to integrate diverse levels of global and local information efficiently. We rigorously validated the proposed method on two distinct datasets, RSSCN7 and RSSRD-KQ, and benchmarked its performance against state-of-the-art SR models. Experiments show that our method achieves SOTA PSNR values of 26.06 and 24.29 on these datasets, respectively, and is visually superior, effectively addressing a variety of scenarios and significantly outperforming existing methods.

A novel pansharpening model based on two parallel network architectures

ABSTRACT Pansharpening is an important technology for obtaining high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) images. Although many pansharpening models have emerged by taking advantage of deep learning (DL) technology, there remains a pressing need to further assess pansharpening accuracy and stability when LRMS images with complex land-cover types. What’s more, these models often overlook the exploitation of PAN images’ inherent high-frequency information. To address these issues, we propose a pansharpening model combining multi-level and multi-scale network architectures. The multi-level network architecture is used to build spatial-spectral dependence on LRMS-PAN pairs, and strengthen the network’s feature capture capability by keeping the multi-level texture details. The multi-scale architecture is subsequently used to extract the spatial structure and deep texture of the PAN images at different scales. Downsampled experiments and real experiments in four standard datasets show that the proposed model achieves a state-of-the-art performance.

Accurate recognition of micromorphology images of epoxy coatings for deep-sea environments based on a deep learning super-resolution method

Crack initiation and extension occur in organic coatings during service in deep-sea environments. However, when extracting detailed crack information from SEM images of epoxy mica coatings at different time periods in a simulated deep-sea fluid-hydraulic environment, uninteresting background regions are treated equally, resulting in unnecessary computational redundancy. To address the relatively blurred edges and unclear textures of SEM images, a crack image super-resolution network based on global mixed attention (GMA-net) is proposed for application to SEM images of organic coatings. The recognition results of the images processed with GMA-net were compared with those of the original images and the images processed with bicubic method, respectively. The results show that this method not only refrains from destroying the clarity of the original images but also greatly outperforms bicubic method in terms of precision, recall, mAP50 and mPA50-95, which are improved by approximately 23.1%, 32.4%, 36.4% and 26.7%, respectively. This method effectively highlights the details and improves the recognition accuracy of the edge texture with the aim of providing a good basis for subsequent recognition and even lifetime prediction studies.

Mapping the Landscape of VFX and AI: A PRISMA-guided Systematic Review

Objective: This systematic literature review aims to comprehensively synthesize existing evidence on Visual Effects (VFX) and Artificial Intelligence (AI) using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The primary objective is to map the landscape of VFX and AI, examining the historical development of VFX techniques and practices in the context of AI advancements. Methods: Following PRISMA guidelines, a thorough search of five electronic databases—IEEE Xplore, Scopus, Taylor & Francis Online, ACM Digital Library, and ScienceDirect—was conducted to identify articles published up to April 18, 2024. Inclusion criteria encompassed English-language journal articles. Independent reviewers screened articles for eligibility, resolving discrepancies through consensus. Data extraction and synthesis were performed systematically. Results: A total of 41 studies met the inclusion criteria and were included in the review. These studies covered a wide range of topics related to VFX and AI, with various study designs. The findings are presented in a narrative synthesis. Discussion: The review indicates that AI development in VFX is comprehensive, covering topics such as image processing, inpainting, video processing, 3D modelling, rendering, deepfake technology, style transfer, tracking, animation, simulation, rigging, texture compression, and depth map enhancement. These advancements highlight the transformative role of AI in enhancing creative expression, production efficiency, and the quality of visual effects. Conclusion: This systematic literature review provides a comprehensive overview of the current state of knowledge on AI and VFX. The integration of AI into VFX is profoundly transforming the industry, enhancing creative expression, production efficiency, and the quality of visual effects. Continued advancements in AI technologies promise to further revolutionize VFX practices, offering new possibilities and efficiencies for future productions.

An underwater image enhancement method based on multi-scale layer decomposition and fusion

High-quality underwater images can intuitively reflect the most realistic underwater conditions, guiding for underwater environmental monitoring and resource exploration strongly. But when factors like light absorption affect underwater optical imaging, it has been found that poor visibility and blurred texture details occur in acquired images, posing challenges for the identification and detection of underwater targets. To obtain natural images, an enhancement algorithm is proposed based on multi-scale layer decomposition and fusion. The algorithm employs different strategies to recover image attenuation information from both local and global perspectives, generating two complementary preprocessed fusion inputs. For fusion input 1, operations are conducted in the RGB color space. Initially, the mean proportion of each color channel is used to identify the attenuated color channel. Then, a local compensation strategy is adaptively applied to restore the pixel intensity of the attenuated color channel. Finally, a statistical color correction method is used to eliminate color cast in the image. Fusion input 2 involves two processing stages. In the Lab color space, the algorithm uses the grayscale information to reduce the deviation in the mean values of channels a and b globally. The local mean information of the component L enhances detail textures. In the RGB color space, linear stretching is applied to correct color deviations. To fuse the structural features of two complementary preprocessed inputs and avoid interference between signals from different layers, the color channels of fused input image are first decomposed into muti-scale structural layers based on structural priors. Then, the image enhancement is achieved through layer-by-layer fusion of the corresponding color channels of the two inputs. By testing and analyzing with the, it was found that the proposed method can improve the clarity of attenuated images in various underwater scenarios of UIEBD and RUIE datasets effectively, enhancing image detail and texture richness, increases contrast, and achieving natural and comfortable visual quality. Compared with the quantitative metrics of 14 other algorithms, the proposed algorithm shows an average score improvement of 10.14, 90.48, and 2.06, respectively, in metrics AG (average gradient), EI (edge intensity), and NIQE (natural image quality evaluator). In the RUIE dataset, it shows an average score improvement of 10.21, 94.76, and 1.86, respectively.

Classification method of seabed sonar image substrate based on ELM-AdaBoost

The results of sediment classification are affected by the measurement environment. The water depth has a significant impact on the sound propagation, and the acoustic characteristics will be different under different water depths. In addition, seabed topography, sediment types and their distribution, acoustic frequency, equipment types and settings, and suspended substances in water will all affect the original sonar data. In order to obtain accurate classification results, a seabed sonar image classification method based on Elm AdaBoost is proposed. The original sound intensity data is compensated for gain and the anomaly is eliminated. After preprocessing the multi beam sonar data, the image texture feature is extracted from the gray level co-occurrence matrix as the feature vector for classification, and the elm classifier is constructed. The elm is enhanced by AdaBoost. The calculated feature is input into the elm AdaBoost network for sediment classification, and the result is output. Simulation results show that the proposed method effectively improves the accuracy of sediment classification, and verifies the feasibility of this method.

NeRF-Texture: Synthesizing Neural Radiance Field Textures.

Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.

A Power Law Reconstruction of Ultrasound Backscatter Images

Ultrasound B-scan images are traditionally formed from the envelope of the received radiofrequency echoes, but the image texture is dominated by granular speckle patterns. Longstanding efforts at speckle reduction and deconvolution have been developed to lessen the detrimental aspects of speckle. However, we now propose an alternative approach to estimation (and image rendering) of the underlying fine grain scattering density of tissues based on power law constraints. The key steps are a whitening of the spectrum of the received signal while conforming to the original envelope shape and statistics, followed by a power law filtering in accordance with the known scattering behavior of tissues. This multiple step approach results in a high-spatial-resolution map of scattering density that is constrained by the most important properties of scattering from tissues. Examples from in vivo liver scans are shown to illustrate the change in image properties from this framework.

Rectified Binary Network for Single-Image Super-Resolution.

Binary neural network (BNN) is an effective approach to reduce the memory usage and the computational complexity of full-precision convolutional neural networks (CNNs), which has been widely used in the field of deep learning. However, there are different properties between BNNs and real-valued models, making it difficult to draw on the experience of CNN composition to develop BNN. In this article, we study the application of binary network to the single-image super-resolution (SISR) task in which the network is trained for restoring original high-resolution (HR) images. Generally, the distribution of features in the network for SISR is more complex than those in recognition models for preserving the abundant image information, e.g., texture, color, and details. To enhance the representation ability of BNN, we explore a novel activation-rectified inference (ARI) module that achieves a more complete representation of features by combining observations from different quantitative perspectives. The activations are divided into several parts with different quantification intervals and are inferred independently. This allows the binary activations to retain more image detail and yield finer inference. In addition, we further propose an adaptive approximation estimator (AAE) for gradually learning the accurate gradient estimation interval in each layer to alleviate the optimization difficulty. Experiments conducted on several benchmarks show that our approach is able to learn a binary SISR model with superior performance over the state-of-the-art methods. The code will be released at https://github.com/jwxintt/Rectified-BSR.

"Identification of Melanoma Cancer Using Feature Extraction from Medical Image Textures by Artificial Intelligence"

"Identification of Melanoma Cancer Using Feature Extraction from Medical Image Textures by Artificial Intelligence"

Capacity and capability of remote sensing to inform invasive plant species management in the Pacific Islands region.

The Pacific Islands region is home to several of the world's biodiversity hotspots, yet its unique flora and fauna are under threat because of biological invasions. These invasions are likely to proliferate as human activity increases and large-scale natural disturbances unfold, exacerbated by climate change. Remote sensing data and techniques provide a feasible method to map and monitor invasive plant species and inform invasive plant species management across the Pacific Islands region. We used case studies taken from literature retrieved from Google Scholar, 3 regional agencies' digital libraries, and 2 online catalogs on invasive plant species management to examine the uptake and challenges faced in the implementation of remote sensing technology in the Pacific region. We synthesized remote sensing techniques and outlined their potential to detect and map invasive plant species based on species phenology, structural characteristics, and image texture algorithms. The application of remote sensing methods to detect invasive plant species was heavily reliant on species ecology, extent of invasion, and available geospatial and remotely sensed image data. However, current mechanisms that support invasive plant species management, including policy frameworks and geospatial data infrastructure, operated in isolation, leading to duplication of efforts and creating unsustainable solutions for the region. For remote sensing to support invasive plant species management in the region, key stakeholders including conservation managers, researchers, and practitioners; funding agencies; and regional organizations must invest, where possible, in the broader geospatial and environmental sector, integrate, and streamline policies and improve capacity and technology access.

Planetary gearbox fault classification based on tooth root strain and GAF pseudo images

Traditional signal processing methods based on acceleration signals can determine whether a fault has occurred in a planetary gearbox. However, acceleration signals are severely affected by interference, causing difficulties in fault identification. This study proposes a gear fault classification method based on root strain and pseudo images. Firstly, fiber optic sensors are employed to directly acquire strain data from the ring gear root. Next, the strain signals are preprocessed using resampling and a time-domain synchronous averaging algorithm. The processed signals are encoded into two-dimensional images using Gramian Angular Fields (GAF). Then, CN-EfficientNet with contrast learning is proposed to analyze and extract deeper fault features from the image texture features. In the classification experiments for different types of faults, the accuracy reached 96.84%. The results indicate that the method can effectively accomplish the task of fault classification in planetary gearboxes. Comparative experiments with other common classification models further indicate the superior performance of the proposed learning model. Visualization based on Grad-CAM provides interpretability for the fault recognition network’s results and reveals the underlying mechanism for its excellent classification performance.

UnWarpME: Unsupervised warping map estimation in real-world scenarios

This paper introduces a novel approach to warp an RGB image to a new target pose, where it learns to inpaint the invisible parts of the image by sampling pixels from the visible parts. This technique is particularly relevant in applications such as novel pose image generation, where the quality of the generated samples heavily relies on the warped images. Conventional methods typically utilize an affine warping map estimator and learn to estimate the warping map through a downstream image generation task. However, this approach results in an affine function being returned regardless of the complexity of the deformation between the source and target poses. In contrast, our proposed method involves estimating the warping map using a convolutional function, which learns through alternating between two downstream tasks. Initially, the estimation treats the warping as part of an image generation task, similar to existing methods but without the constraint of an affine transformation. This encourages the estimation of complex transformations beyond affine deformations. Subsequently, disregarding the image generation task, we supervise the convolutional warping map to learn any potential affine transformation between the guiding poses of the sample. Our experimental results demonstrate the high effectiveness of our method in preserving image textures while transforming it into highly complex poses.

Comparative Study of Denoising and Segmentation Techniques for Accurate Brain Tumor Detection in MRI

Background: Brain tumor incidence is on the rise each year, with more than 130 identified types. Precise segmentation models play a vital role in the diagnosis and treatment of brain tumors. This study specifically investigates the utilization of diffusion-based denoising techniques and thresholding methods for segmenting brain tumors from MRI images. Objective: The objective of this study is to examine and compare the efficacy of the Perona-malik and Weickert diffusion techniques in denoising brain MRI images. Additionally, the study aims to assess their performance in threshold-based segmentation of brain tumors. Moreover, it also aims to evaluate the compatibility, benefits, and limitations of the Perona-Malik and Weickert diffusion methods in the denoising of brain MRI images and the effect of denoising on segmentation. Methods: In this study, the Perona-Malik and Weickert diffusion methods are employed to denoise brain MRI images. The denoised images are then subjected to thresholding using both binary and fuzzy approaches, utilizing a triangular membership function. The performance of the diffusion techniques is evaluated using metrics, such as Mean Square Error and Peak Signal to Noise Ratio. Additionally, segmentation models are assessed using metrics such as Dice Similarity Coefficient, Jaccard Similarity Coefficient, and Structural Similarity Measurement Index. Results: The Perona-Malik and Weickert diffusion methods exhibit compatibility with various types of noise, each having its own set of advantages and limitations. The Weickert diffusion method excels in preserving image structure and texture during thresholding. Conclusion: The study provides evidence for the effectiveness of diffusion-based denoising techniques in segmenting brain tumors from MRI images. Specifically, the Weickert diffusion method outperforms in preserving essential image characteristics during thresholding. Additionally, fuzzy thresholding proves to be more successful in accurately segmenting brain tumors. These findings contribute to the advancement of precise models for brain tumor segmentation, ultimately enhancing the diagnosis and treatment of these tumors.

Texture-based speciation of otitis media-related bacterial biofilms from optical coherence tomography images using supervised classification.

Otitis media (OM), a highly prevalent inflammatory middle-ear disease in children worldwide, is commonly caused by an infection, and can lead to antibiotic-resistant bacterial biofilms in recurrent/chronic OM cases. A biofilm related to OM typically contains one or multiple bacterial species. OCT has been used clinically to visualize the presence of bacterial biofilms in the middle ear. This study used OCT to compare microstructural image texture features from bacterial biofilms. The proposed method applied supervised machine-learning-based frameworks (SVM, random forest, and XGBoost) to classify multiple species bacterial biofilms from in vitro cultures and clinically-obtained in vivo images from human subjects. Our findings show that optimized SVM-RBF and XGBoost classifiers achieved more than 95% of AUC, detecting each biofilm class. These results demonstrate the potential for differentiating OM-causing bacterial biofilms through texture analysis of OCT images and a machine-learning framework, offering valuable insights for real-time in vivo characterization of ear infections.

Fast high quality highlight removal using a simplified matrix iteration method

Recently, a global non-smooth optimization method was developed for efficiently eliminating specular highlights from single color image. Although this method was reported to be effective on both natural and medical images, its algorithm’s speed is slow due to the non-smoothness of the objective function. This paper proposes a global smooth optimization method, where two gradient regularization terms are used to describe diffuse and specular component features for texture details. A simplified optimality condition is established and thus a new iteration algorithm for specular highlight removal is proposed. By employing the gradient regularization of diffuse reflection, image texture detail can be preserved during specular highlight removal. Due to the smoothness of the objective function, the proposed smooth optimization algorithm has lower computational complexity and less storage requirement than the global non-smooth optimization algorithm. Furthermore, the proposed iteration algorithm is guaranteed to converge to a global optimal solution under a fixed step length. Experimental results demonstrate that the proposed smooth optimization algorithm is much faster than the non-smooth optimization algorithm. Moreover, the proposed highlight removal algorithm is more effective on benchmark natural images and real laparoscopic images than conventional highlight removal algorithms.

Infrared and visible image fusion method based on target enhancement and rat swarm optimization

In order to solve the target ambiguity and information loss in the fusion results of traditional infrared and visible images, a fusion method of infrared and visible images based on the target enhancement and mouse swarm optimization, which is abbreviated as TERSFuse. Firstly, in order to reduce the loss of the original image details in the fusion results, the infrared contrast enhancement module and the visible image enhancement module based on the brightness perception are constructed respectively. Secondly, the infrared and visible enhanced images were decomposed by using the Laplace pyramid transform to obtain the corresponding high and low frequency images. In order to make the fusion result fully retain the original information, the "maximum absolute value" rule is used to fuse the infrared and visible high frequency images, and the low frequency images are fused by calculating the weight coefficient. Finally, the image reconstruction module based on the rat swarm optimization is designed to achieve the adaptive allocation of weight parameters of high frequency and low frequency image reconstruction, and then improve the visual effect of the fused image. In order to verify the advantages of the present algorithm, the experimental results show that the present algorithm not only obtains the good visual effects, but also can retains the rich edge texture and contrast information of the original image.

A fuzzy control algorithm based on artificial intelligence for the fusion of traditional Chinese painting and AI painting

Recently, artificial intelligence (AI)-generated resources have gained popularity because of their high effectiveness and reliability in terms of output and capacity to be customized and broadened, especially in image generation. Traditional Chinese paintings (TCPs) are incomplete because their color contrast is insufficient, and object reality is minimal. However, combining AI painting (AIP) with TCP remains inadequate and uncertain because of image features such as patterns, styles, and color. Hence, an algorithm named variational fusion-based fuzzy accelerated painting (VF2AP) has been proposed to resolve this challenge. Initially, the collected TCP data source is applied for preprocessing to convert it into a grayscale image. Then, the feature extraction process is performed via fuzzy-based local binary pattern (FLBP) and brushstroke patterns to enhance the fusion of intelligent fuzzy logic to optimize the local patterns of textures in a noisy image. Second, the extracted features are used as inputs to the variational autoencoder (VAE), which is used to avoid latent space irregularities in the image and the reconstructed image by maintaining minimum reconstruction loss. Third, fuzzy inference rules are applied to avoid variation in the fusion process of the reconstructed and original images. Fourth, the feedback mechanism is designed with evaluation metrics such as area under the curve-receiver operating characteristic (AUC-ROC) analysis, mean square error (MSE), structural similarity index (SSIM), and Kullback‒Leibler (KL) divergence to enhance the viewer's understanding of fused painting images.

Boundary-Aware Gradient Operator Network for Medical Image Segmentation.

Medical image segmentation is a crucial task in computer-aided diagnosis. Although convolutional neural networks (CNNs) have made significant progress in the field of medical image segmentation, the convolution kernels of CNNs are optimized from random initialization without explicitly encoding gradient information, leading to a lack of specificity for certain features, such as blurred boundary features. Furthermore, the frequently applied down-sampling operation also loses the fine structural features in shallow layers. Therefore, we propose a boundary-aware gradient operator network (BG-Net) for medical image segmentation, in which the gradient convolution (GConv) and the boundary-aware mechanism (BAM) modules are developed to simulate image boundary features and the remote dependencies between channels. The GConv module transforms the gradient operator into a convolutional operation that can extract gradient features; it attempts to extract more features such as images boundaries and textures, thereby fully utilizing limited input to capture more features representing boundaries. In addition, the BAM can increase the amount of global contextual information while suppressing invalid information by focusing on feature dependencies and the weight ratios between channels. Thus, the boundary perception ability of BG-Net is improved. Finally, we use a multi-modal fusion mechanism to effectively fuse lightweight gradient convolution and U-shaped branch features into a multilevel feature, enabling global dependencies and low-level spatial details to be effectively captured in a shallower manner. We conduct extensive experiments on eight datasets that broadly cover medical images to evaluate the effectiveness of the proposed BG-Net. The experimental results demonstrate that BG-Net outperforms the state-of-the-art methods, particularly those focused on boundary segmentation.

Invisible gas detection: An RGB-thermal cross attention network and a new benchmark

The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high toxicity. Thermal infrared-based computer vision detection techniques provide a straightforward approach to identify gas leakage areas. However, the development of high-quality algorithms has been challenging due to the low texture in thermal images and the lack of open-source datasets. In this paper, we present the RGB-Thermal Cross Attention Network (RT-CAN), which employs an RGB-assisted two-stream network architecture to integrate texture information from RGB images and gas area information from thermal images. Additionally, to facilitate the research of invisible gas detection, we introduce Gas-DB, an extensive open-source gas detection database including about 1.3K well-annotated RGB-thermal images with eight variant collection scenes. Experimental results demonstrate that our method successfully leverages the advantages of both modalities, achieving state-of-the-art (SOTA) performance among RGB-thermal methods, surpassing single-stream SOTA models in terms of accuracy, Intersection of Union (IoU), and F2 metrics by 4.86%, 5.65%, and 4.88%, respectively. The code and data can be found at https://github.com/logic112358/RT-CAN.