Image Transformation Research Articles

Texture Analysis and Classification using Local Binary Patterns and Statistical Features

Texture analysis and classification are crucial in many applications such as medical applications, satellite imagery analysis, and so on. This paper presents a new technique that incorporates the LBP with the HOG to improve the texture analysis and classification. LBP is famous for finding out the local texture descriptors by comparing the pixel intensities it deals with and can challenge light variations and image transformations. On the other hand, there is HOG which emphasizes edge and gradient information to detect the orientation and shape of a texture in a given image. While LBP describes the local micro-patterns HOG gives the perspective of the global gradient orientation. The combined use of local and global texture information is helpful in the presented problem when prioritizing the recognition of complex texture patterns that cannot be deciphered by a single algorithm. To demonstrate the efficiency of the proposed hybrid method, several experiments are performed on several texture databases. According to the findings, the proposed LBP-HOG strategy proves more beneficial than basic LBP and HOG techniques concerning the classification rate and computing time. The combination method shows improved performance in terms of agreement with the reference of fine and coarse level of texture descriptions along with providing large-scale description of the texture which results in the improved discriminatory power of the texture classes.

Multi-Temporal Snow-Covered Remote Sensing Image Matching via Image Transformation and Multi-Level Feature Extraction

To address the challenge of image matching posed by significant modal differences in remote sensing images influenced by snow cover, this paper proposes an innovative image transformation-based matching method. Initially, the Pix2Pix-GAN conversion network is employed to transform remote sensing images with snow cover into images without snow cover, reducing the feature disparity between the images. This conversion facilitates the extraction of more discernible features for matching by transforming the problem from snow-covered to snow-free images. Subsequently, a multi-level feature extraction network is utilized to extract multi-level feature descriptors from the transformed images. Keypoints are derived from these descriptors, enabling effective feature matching. Finally, the matching results are mapped back onto the original snow-covered remote sensing images. The proposed method was compared to well-established techniques such as SIFT, RIFT2, R2D2, and ReDFeat and demonstrated outstanding performance. In terms of NCM, MP, Rep, Recall, and F1-measure, our method outperformed the state of the art by 177, 0.29, 0.22, 0.21, and 0.25, respectively. In addition, the algorithm shows robustness over a range of image rotation angles from −40° to 40°. This innovative approach offers a new perspective on the task of matching multi-temporal snow-covered remote sensing images.

Bilateral fusion low‐light image enhancement with implicit information constraints

AbstractResearch on low‐light image enhancement focuses on improving image quality in dim conditions. Recently, deep learning has driven significant advancements, with many studies using neural networks to enhance low‐light images. However, most focus on complex network designs to increase nonlinearity, often neglecting the implicit information from local image transformations. This paper introduces an improved U‐net‐based method for low‐light enhancement, retaining the original encoding network and adding branch links in the decoding network. The method uses attention feature fusion to handle image noise and gradients separately, adjusting brightness through a gradient‐adaptive transform. This approach optimizes performance using loss functions like peak signal‐to‐noise ratio and colour consistency. Unlike previous methods, the approach emphasizes extracting implicit information from image gradients, achieving enhancement that aligns with the original brightness distribution. The result is enhanced images with high detail similarity to the original, achieved through end‐to‐end inference in experiments.

Multi-spectral image transformer descriptor classification combined with molecular tools for early detection of tomato grey mould

Accurate early detection of plant diseases is a critical milestone in the development of self-navigating robots for precision agriculture and still remains one of the most significant challenges. Plant pathogens, like Botrytis cinerea causing grey mould disease, pose significant threats to agriculture and food safety. This study focuses on the early detection of B. cinerea in tomato crops using Artificial Intelligence. Specifically, the Deep Learning (DL) YOLOv8 architecture was employed to apply single class segmentation on plant captures to extract the tomato leaves locational information. Each plant capture includes five images from hyperspectral wavelengths (at 460, 540, 640, 775 and 875 nm) and one RGB images. The leaf segmentation achieved 81.7 % Mean Average Precision (mAP) at Intersection over Union (IoU) threshold 0.5 (mAP50). Then the for each leaf segment, and array of descriptors is extracted through various Transformer models. Finally, the descriptors are classified through a KNN or an LSTM solution and the results of all the descriptors for each leaf from each of the Transformer models, for each of the six images of the capture, are ensembled to yield the class of each leaf. The best ensemble results were produced by only accumulating results from some of the Transformer and specifically the MaxViT-L, the ViT-B (P:16 × 16 – C), the VOLO (D:5) and the XCIT-L (L:24 – P:16 × 16), by only using the results provided by the images at the hyperspectral wavelengths at 540, 640 nm and the RGB image, and by applying the LSTM solution. The process achieved a 79.41 % classification accuracy and 71.12 % F1-Score for all classes. Additionally, early, rapid and accurate detection of grey mould at the early stages of the infection or at latent infections when symptoms are not visible was assessed by comparative qPCR-based methods (qRT-PCR) for fungal biomass estimation in plant tissues and the rest of this study's methodology. The findings of this study represent a significant advancement in crop disease detection and management, and highlight the potential for integrating these methods into on-site digital systems (robots, mobile apps, etc.) under real-world settings. The results, demonstrate the effectiveness of combining segmentation and transformer-based classification models for early and accurate detection of grey mould disease.

On image transformation for partial discharge source identification in vehicle cable terminals of high‐speed trains

AbstractPartial discharge (PD) detection of cable terminals is crucial for the safe operation of the traction power system in trains. However, similar PD signals in complex train‐operating environments cause difficulty to recognise the insulation defects. Therefore, a PD signal image transformation recognition method is proposed for PD detection of cable terminal defects to identify defects in cable terminals with similar PD characteristics accurately. In the proposed method, the raw PD signals are firstly transformed to images via the Gramian angular field (GAF) representation. This can reveal the discriminative characteristics embedded in the original PD signals and subsequently facilitate differentiating the PD sources, which exhibit similar characteristic in the time domain. The obtained GAF representation of PD signals (named as PD GAF images) is extracted from local and global features to train an efficient MobileVIT model, which is then utilised to identify similar types of PD sources in cable terminals. The results show that the proposed method achieves 97.5% recognition accuracy in the field experiment, which is superior to other methods.

Lightweight vision image transformer (LViT) model for skin cancer disease classification

Lightweight vision image transformer (LViT) model for skin cancer disease classification

Learning to reconstruct accelerated MRI through K-space cold diffusion without noise

Deep learning-based MRI reconstruction models have achieved superior performance these days. Most recently, diffusion models have shown remarkable performance in image generation, in-painting, super-resolution, image editing and more. As a generalized diffusion model, cold diffusion further broadens the scope and considers models built around arbitrary image transformations such as blurring, down-sampling, etc. In this paper, we propose a k-space cold diffusion model that performs image degradation and restoration in k-space without the need for Gaussian noise. We provide comparisons with multiple deep learning-based MRI reconstruction models and perform tests on a well-known large open-source MRI dataset. Our results show that this novel way of performing degradation can generate high-quality reconstruction images for accelerated MRI.

Identification of the damage location for the structural sealant based on deep learning

Structural sealants are essential to maintain the safety of panel units of hidden frame glass curtain wall. Damage localization of the concealed structural sealant is still difficult because of the unknown baseline model. In this paper, a convolutional neural network-based identification method is proposed to localize the damage of structural sealants without the baseline model. The method develops a novel input of the convolutional neural network (CNN), multi-symmetry-point images (MSPI), which is encoded by the response of four symmetrical points to impulse excitations. The CNN would identify the damage location by discerning differences among four images. Then, a dataset with 28803 samples, which considers the effect of the multiple damages and noise, was used to train the CNN. Three types of transformed images and four CNN models were compared to optimize the input signals and the configuration of the CNN. A series of numerical examples indicated that the Gram angle difference field is the optimal image transformation method, and improved-DenseNet121 is the optimal CNN for damage localization in structural sealants. Then, several laboratory experiments validate the effectiveness of the optimized image input and CNN with an accuracy rate of 92 % in the identification of damage location.

Early Hotspot Detection in Photovoltaic Modules using Deep Learning Methods

Photovoltaic (PV) energy systems have been widely used in energy production especially in recent years due to their clean, reliable, environmentally friendly and resource continuity. The electrical energy to be obtained must be uninterrupted and of high quality. For this reason, faults that may cause production losses in PV power plants should be detected quickly and accurately. In this study, an approach that can automatically detect hotspot fault in solar PV modules is presented. In the proposed approach, firstly, Jet colormap based colored image transformation process is applied to gray scale infrared image data. Then, using three different deep learning models such as MobilNetV2, ResNet-18, InceptionV3, both original grey scale data and Jet colormap data are detected and evaluation metric values are obtained. The importance of this research lies in the potential of deep learning-based effective detection, especially in the early and rapid diagnosis of PV hotspot faults.

Efficient visual transformer transferring from neural ODE perspective

AbstractRecently, the Visual Image Transformer (ViT) has revolutionized various domains in computer vision. The transfer of pre‐trained ViT models on large‐scale datasets has proven to be a promising method for downstream tasks. However, traditional transfer methods introduce numerous additional parameters in transformer blocks, posing new challenges in learning downstream tasks. This article proposes an efficient transfer method from the perspective of neural Ordinary Differential Equations (ODEs) to address this issue. On the one hand, the residual connections in the transformer layers can be interpreted as the numerical integration of differential equations. Therefore, the transformer block can be described as two explicit Euler method equations. By dynamically learning the step size in the explicit Euler equation, a highly lightweight method for transferring the transformer block is obtained. On the other hand, a new learnable neural memory ODE block is proposed by taking inspiration from the self‐inhibition mechanism in neural systems. It increases the diversity of dynamical behaviours of the neurons to transfer the head block efficiently and enhances non‐linearity simultaneously. Experimental results in image classification demonstrate that the proposed approach can effectively transfer ViT models and outperform state‐of‐the‐art methods.

Training Artificial Neural Networks to Detect Multiple Sclerosis Lesions Using Granulometric Data from Preprocessed Magnetic Resonance Images with Morphological Transformations

The symptoms of multiple sclerosis (MS) are determined by the location of demyelinating lesions in the white matter of the brain and spinal cord. Currently, magnetic resonance imaging (MRI) is the most common tool used for diagnosing MS, understanding the course of the disease, and analyzing the effects of treatments. However, undesirable components may appear during the generation of MRI scans, such as noise or intensity variations. Mathematical morphology (MM) is a powerful image analysis technique that helps to filter the image and extract relevant structures. Granulometry is an image measurement tool for measuring MM that determines the size distribution of objects in an image without explicitly segmenting each object. While several methods have been proposed for the automatic segmentation of MS lesions in MRI scans, in some cases, only simple data preprocessing, such as image resizing to standardize the input dimensions, has been performed before the algorithm training. Therefore, this paper proposes an MRI preprocessing algorithm capable of performing elementary morphological transformations in brain images of MS patients and healthy individuals in order to delete undesirable components and extract the relevant structures such as MS lesions. Also, the algorithm computes the granulometry in MRI scans to describe the size qualities of lesions. Using this algorithm, we trained two artificial neural networks (ANNs) to predict MS diagnoses. By computing the differences in granulometry measurements between an image with MS lesions and a reference image (without lesions), we determined the size characterization of the lesions. Then, the ANNs were evaluated with the validation set, and the performance results (test accuracy = 0.9753; cross-entropy loss = 0.0247) show that the proposed algorithm can support specialists in making decisions to diagnose MS and estimating the disease progress based on granulometry values.

Религия – политика – нация: трансформация конфессиональных образов Тридцатилетней войны (1618-1648)

The article is devoted to the transformation of confessional images into national ones in German visual journalism during the Thirty Years' War (1618-1648). This is most clearly seen in a unique German monument from the Thirty Years' War era - illustrated single-leaf woodcuts on which images were combined with text below. The engravings of the illustrated single-leaf woodcuts reflected all the most pressing processes and events of the early modern period, including the crisis of religious consciousness and the collapse of the idea of a “universal monarchy” - a single Christian state under the auspices of the Holy Roman Emperor. During the era of the Thirty Years' War, politics could not be imagined without religion, and religious motives were not separated from political ones. The Thirty Years' War, which turned into the first pan-European catastrophe, became a new step in the secularization of consciousness, which contributed to the metamorphosis of political teachings of the 17th century. The Thirty Years' War, which turned into the first pan-European catastrophe, became a new step in the secularization of consciousness, which contributed to the metamorphosis of political teachings of the 17th century. The gap between politics and religion that began at this time was expressed, among other things, in the emergence of the idea of the nation as one of the leading leitmotifs of state development. National consciousness gradually turned into a kind of foundation for politics. During the era of the Thirty Years' War, this was reflected in the interpretation of Christian symbols and confessional images in visual media. In illustrated single-leaf woodcuts, they were increasingly associated with claims to the uniqueness of one’s own nation and the need for state sovereignty. Traditional religious meanings and motives were used to create images of both national heroes and “anti-heroes” - enemies of the state and nation.

Unrecognizable yet identifiable: Image distortion with preserved embeddings

Biometric authentication systems play a crucial role in modern security systems. However, maintaining the balance of privacy and integrity of stored biometrics derivative data while achieving high recognition accuracy is often challenging. Addressing this issue, we introduce an innovative image transformation technique that effectively renders facial images unrecognizable to the eye while maintaining their identifiability by neural network models, which allows the distorted photo version to be stored for further verification. While initially intended for biometrics systems, the proposed methodology can be used in various artificial intelligence applications to distort the visual data and keep the derived features close. By experimenting with widely used datasets LFW and MNIST, we show that it is possible to build the distortion that changes the image content by more than 70% while maintaining the same recognition accuracy. We compare our method with previously state-of-the-art approaches. We publically release the source code.11https://github.com/ZamDimon/distortion-generator/tree/v1.0.0.

Explore the Transformation Design of Mount Tai Culture IP Image

Mount Tai, with its long history and rich cultural heritage, is China's first dual heritage site of culture and nature. With the trend of cultural and tourism integration and the booming cultural and intellectual property (IP) industry, Mount Tai culture urgently requires IP image development. This paper, by reviewing Mount Tai's cultural resources, explores the patterns of transforming Mount Tai culture into IP images, aiming to provide a research basis for the transformation of IP images of other cultural and tourism resources and to better promote the development of the cultural and tourism industry.

Deep learning for mango leaf disease identification: A vision transformer perspective

Over the last decade, the use of machine learning in smart agriculture has surged in popularity. Deep learning, particularly Convolutional Neural Networks (CNNs), has been useful in identifying diseases in plants at an early stage. Recently, Vision Transformers (ViTs) have proven to be effective in image classification tasks. These architectures often outperform most state-of-the-art CNN models. However, the adoption of vision transformers in agriculture is still in its infancy. In this paper, we evaluated the performance of vision transformers in identification of mango leaf diseases and compare them with popular CNNs. We proposed an optimized model based on a pretrained Data-efficient Image Transformer (DeiT) architecture that achieves 99.75% accuracy, better than many popular CNNs including SqueezeNet, ShuffleNet, EfficientNet, DenseNet121, and MobileNet. We also demonstrated that vision transformers can have a shorter training time than CNNs, as they require fewer epochs to achieve optimal results. We also proposed a mobile app that uses the model as a backend to identify mango leaf diseases in real-time.

Artificial intelligence-empowered assessment of bile duct stone removal challenges

The aim of this investigation was to unravel the complexities inherent in endoscopic retrograde cholangiopancreatography (ERCP) procedures for bile duct stone removal by leveraging advanced artificial intelligence (AI) methodologies to support clinical decision-making and optimize patient outcomes. We introduced the Assessment of Stone Removal Challenges (ACRS) system, a novel integration of Data-efficient Image Transformers (DeiT) for image data analysis and eXtreme Gradient Boosting (XGBoost) for clinical data interpretation. Our study included a patient cohort of 2,129 individuals, focusing on training the ACRS system to achieve high diagnostic precision. Using logistic regression, we identified pivotal predictors affecting the complexity of bile duct stone removal. These findings were visually represented through forest plots and nomograms. Analytical and visualization processes were conducted using the Python and R programming languages, adhering to a p value significance threshold of less than 0.05. By utilizing DeiT enhanced by transfer learning and XGBoost for clinical data interpretation, the system achieved an accuracy of 0.83 and a perfect recall rate on a test set of 2,129 patients. Gradient-weighted class activation mapping (Grad-CAM) and SHapley Additive exPlanations (SHAP) provided in-depth diagnostic insights. Logistic regression was applied to identify crucial clinical predictors for stone removal difficulty, as visualized through forest plots and nomograms. These tools facilitated measurable assessments of procedural complexity, making significant strides in gastroenterology diagnostics and decision-making. By adopting causal inference techniques, the system effectively quantifies the influence of various clinical entities on stone removal difficulty, thereby augmenting both diagnostic precision and procedural strategies in ERCP.

Dual‐task enhanced global–local temporal–spatial network for depression recognition from facial videos

SummaryIn previous studies on facial video depression recognition, although convolutional neural network (CNN) has become a mainstream method, its performance still has room for improvement due to the insufficient extraction of global and local information and the neglect of the correlation of temporal and spatial information. This paper proposes a novel dual‐task enhanced global–local temporal–spatial network (DTE‐GLTS) to enhance the extraction capability of global and local features and deepen the analysis of temporal–spatial information correlation. We design a dual‐task learning mode that utilizes the data‐efficient image transformer (Deit) as the main body to learn the global features of video sequences and guides Deit to learn local features with the pre‐trained temporal–spatial fusion network (TSF). In addition, we propose the TSF mechanism to more effectively fuse temporal–spatial information in video sequences, strengthen the correlation between frames and pixels, and embed it in Resnet to form the TSF network. To the best of our knowledge, this is the first application of Deit and dual‐task learning mode in the field of facial video depression recognition. The experimental results on AVEC 2013 and AVEC 2014 show that our method achieves competitive performance, with mean absolute error/root mean square error (MAE/RMSE) scores of 6.06/7.73 and 5.91/7.68, respectively, while significantly reducing the number of parameters.

Multi-modal transformer architecture for medical image analysis and automated report generation

Medical practitioners examine medical images, such as X-rays, write reports based on the findings, and provide conclusive statements. Manual interpretation of the results and report generation by examiners are time-consuming processes that lead to potential delays in diagnosis. We propose an automated report generation model for medical images leveraging an encoder–decoder architecture. Our model utilizes transformer architectures, including Vision Transformer (ViT) and its variants like Data Efficient Image Transformer (DEiT) and BERT pre-training image transformer (BEiT), as an encoder. These transformers are adapted for processing to extract and gain visual information from medical images. Reports are transformed into text embeddings, and the Generative Pre-trained Transformer (GPT2) model is used as a decoder to generate medical reports. Our model utilizes a cross-attention mechanism between the vision transformer and GPT2, which enables it to create detailed and coherent medical reports based on the visual information extracted by the encoder. In our model, we have extended the report generation with general knowledge, which is independent of the inputs and provides a comprehensive report in a broad sense. We conduct our experiments on the Indiana University X-ray dataset to demonstrate the effectiveness of our models. Generated medical reports from the model are evaluated using word overlap metrics such as Bleu scores, Rouge-L, retrieval augmentation answer correctness, and similarity metrics such as skip thought cs, greedy matching, vector extrema, and RAG answer similarity. Results show that our model is performing better than the recurrent models in terms of report generation, answer similarity, and word overlap metrics. By automating the report generation process and incorporating advanced transformer architectures and general knowledge, our approach has the potential to significantly improve the efficiency and accuracy of medical image analysis and report generation.

Classification and identification of extreme wind events by CNNs based on Shapelets and improved GASF-GADF

In this manuscript, we propose an automatic classification and recognition method for extreme wind events based on Convolutional Neural Networks (CNNs) and combining the Shapelet Transform (ST) algorithm with the improved Gramian Angular Summation Field - Gramian Angular Difference Field (GASF-GADF) 2D images construction format. First, a CNN model suitable for wind speed time series 2D images classification and recognition among five mainstream CNNs (ResNet-50, ShuffleNet0.5 × , DenseNet-121, EfficientNet-B2, and EfficientNetV2-S) is preferred based on the basic Gramian Angular Field (GAF) method; then, the improved GASF-GADF images construction format is proposed, and the optimal CNN is used to compare the classification and recognition results based on other three image conversion methods: Markov Transition Field (MTF), GASF, GADF. Last, it is proposed to utilize the ST algorithm to extract the feature subsequence Shapelets of wind speed time series to further improve the classification and recognition effect on extreme wind events. The effectiveness and applicability of the proposed method were verified through three extreme wind event datasets in this paper.The results show that the combination of Shapelets and the improved GASF-GADF images transformation method proposed in this paper can effectively enhance the classification and recognition of extreme wind events by CNNs. Among them, EfficientNetV2-S combined with the method proposed in this paper achieves 99.50%, 99.50% and 97.50% recognition Accuracy for thunderstorm, gust front and typhoon, respectively. Meanwhile, it still has good robustness for extreme wind events disturbed by noise.

Research on airfoil surface flow structure testing based on Tuft velocity measurement method

Fluorescent tuft visualization technology, as a practical and convenient technique, has attracted widespread attention. In this study, a novel fluorescent tuft velocity measurement method is developed based on existing technologies. This method involves the recognition of tuft deflection angles, image transformation, block processing, temporal averaging, and post-visualization processing. Additionally, quantitative measurement of flow field velocity is achieved through tuft calibration. The study begins by comparing tufts of different materials, examining both fluorescent and deflection characteristics to determine optimal tuft parameters. The impact of tuft length on deflection characteristics is also investigated. Finally, the cotton tuft with a length of 7 mm and a diameter of about 0.1 mm was obtained as the best tuft sought in this experiment. Subsequently, at Reynolds number Re = 1.7 × 105, airfoil surface flow field visualization and quantitative analysis are conducted, exploring the relationship between airfoil surface flow structure and aerodynamics at different angles of attack. Finally, a comparative study is conducted between tuft velocity results and oil flow visualization experiments, as well as particle image velocimetry experiments, confirming the feasibility of the fluorescent tuft velocity measurement method.