Image Converter Research Articles

Cartoonization of Images Using Generative Adversarial Network

Abstract: This research paper introduces a web-based image style conversion system powered by a trained AnimeGANv2 model. The system enables real-time conversion of images into anime-style artwork within a web application. Through the integration of TensorFlow, Protobuf, and ONNX formats, the AnimeGANv2 model is seamlessly deployed for efficient style conversion. The web application incorporates user authentication, secure image uploading, and instant style conversion features. Evaluation through user testing assesses the system's usability, performance, and user satisfaction. Results demonstrate the successful implementation of real- time image style conversion, offering a practical solution for users interested in artistic rendering. This research contributes to understanding GAN-based image style conversion applications and their integration into web platforms for creative expression and visual content generation.

Semi-supervised segmentation of metal-artifact contaminated industrial CT images using improved CycleGAN.

Accurate segmentation of industrial CT images is of great significance in industrial fields such as quality inspection and defect analysis. However, reconstruction of industrial CT images often suffers from typical metal artifacts caused by factors like beam hardening, scattering, statistical noise, and partial volume effects. Traditional segmentation methods are difficult to achieve precise segmentation of CT images mainly due to the presence of these metal artifacts. Furthermore, acquiring paired CT image data required by fully supervised networks proves to be extremely challenging. To address these issues, this paper introduces an improved CycleGAN approach for achieving semi-supervised segmentation of industrial CT images. This method not only eliminates the need for removing metal artifacts and noise, but also enables the direct conversion of metal artifact-contaminated images into segmented images without the requirement of paired data. The average values of quantitative assessment of image segmentation performance can reach 0.96645 for Dice Similarity Coefficient(Dice) and 0.93718 for Intersection over Union(IoU). In comparison to traditional segmentation methods, it presents significant improvements in both quantitative metrics and visual quality, provides valuable insights for further research.

Optimized Mirror Generative Adversarial Network with BERT Neural Architecture for Text Caption to Image Conversion

Optimized Mirror Generative Adversarial Network with BERT Neural Architecture for Text Caption to Image Conversion

An improvement on common optimization methods based on SuperstarGAN

Image processing has long been a focal point of research, offering avenues to enhance image clarity and transfer image features. Over the past decade, Generative Adversarial Networks (GANs) have played a pivotal role in the field of image conversion. This study delves into the world of GANs, focusing on the SuperstarGAN model and its optimization techniques. SuperstarGAN, an evolution of the well-known StarGAN, excels in multi-domain image-to-image conversion, overcoming limitations and offering versatility. To better understand its optimization, this study explored the effects of different optimizers, such as Adam, SGD, and Nadam, on SuperstarGAN's performance. Using the CelebA Face Dataset with 200 million images and 40 features, this study conducted experiments to compare these optimizers. The results revealed that while SGD and Nadam can achieve comparable results to Adam, they require more iterations and careful tuning, with SGD showing slower convergence. Nadam, with its oscillatory nature, shows promise but requires proper learning rate adjustments. This research sheds light on the critical role of optimizer choice in training SuperstarGAN. Adam emerges as the most efficient and stable option, but further exploration of Nadam's potential is warranted. This study contributes to advancing the understanding of optimization techniques for generative adversarial networks, with implications for high-quality facial image generation and beyond.

Optimize & Reduce: A Top-Down Approach for Image Vectorization

Vector image representation is a popular choice when editability and flexibility in resolution are desired. However, most images are only available in raster form, making raster-to-vector image conversion (vectorization) an important task. Classical methods for vectorization are either domain-specific or yield an abundance of shapes which limits editability and interpretability. Learning-based methods, that use differentiable rendering, have revolutionized vectorization, at the cost of poor generalization to out-of-training distribution domains, and optimization-based counterparts are either slow or produce non-editable and redundant shapes. In this work, we propose Optimize & Reduce (O&R), a top-down approach to vectorization that is both fast and domain-agnostic. O&R aims to attain a compact representation of input images by iteratively optimizing Bezier curve parameters and significantly reducing the number of shapes, using a devised importance measure. We contribute a benchmark of five datasets comprising images from a broad spectrum of image complexities - from emojis to natural-like images. Through extensive experiments on hundreds of images, we demonstrate that our method is domain agnostic and outperforms existing works in both reconstruction and perceptual quality for a fixed number of shapes. Moreover, we show that our algorithm is x10 faster than the state-of-the-art optimization-based method. Our code is publicly available: https://github.com/ajevnisek/optimize-and-reduce

Anomaly‐based Alzheimer's disease detection using entropy‐based probability Positron Emission Tomography images

AbstractDeep neural networks trained on labeled medical data face major challenges owing to the economic costs of data acquisition through expensive medical imaging devices, expert labor for data annotation, and large datasets to achieve optimal model performance. The heterogeneity of diseases, such as Alzheimer's disease, further complicates deep learning because the test cases may substantially differ from the training data, possibly increasing the rate of false positives. We propose a reconstruction‐based self‐supervised anomaly detection model to overcome these challenges. It has a dual‐subnetwork encoder that enhances feature encoding augmented by skip connections to the decoder for improving the gradient flow. The novel encoder captures local and global features to improve image reconstruction. In addition, we introduce an entropy‐based image conversion method. Extensive evaluations show that the proposed model outperforms benchmark models in anomaly detection and classification using an encoder. The supervised and unsupervised models show improved performances when trained with data preprocessed using the proposed image conversion method.

Heart Sound Signals Classification with Image Conversion Employed

The number of patients with cardiovascular diseases worldwide is increasing rapidly, while medical resources are increasingly scarce. Heart sound classification, as the most direct means of discovering cardiovascular diseases, is attracting the attention of researchers around the world. Although great progress has been made in heart sound classification in recent years, most of them are based on traditional statistical feature methods and temporal dimension features. These traditional temporal dimension feature representation and classification methods cannot achieve good classification accuracy. This paper proposes a new partition attention module and Fusionghost module, and the entire network framework is named PANet. Without segmentation of the heart sound signal, the heart sound signal is converted into a bispectrum and input into the proposed framework for feature extraction and classification tasks. The network makes full use of multi-scale feature extraction and feature map fusion, improving the network feature extraction ability. This paper conducts a comprehensive study of the performance of different network parameters and different module numbers, and compares the performance with the most advanced algorithms currently available. Experiments have shown that for two classification problems (normal or abnormal), the classification accuracy rate on the 2016 PhysioNet/CinC Challenge database reached 97.89%, the sensitivity was 96.96%, and the specificity was 98.85%.

Bio-Inspired Dark Adaptive Nighttime Object Detection.

Nighttime object detection is challenging due to dim, uneven lighting. The IIHS research conducted in 2022 shows that pedestrian anti-collision systems are less effective at night. Common solutions utilize costly sensors, such as thermal imaging and LiDAR, aiming for highly accurate detection. Conversely, this study employs a low-cost 2D image approach to address the problem by drawing inspiration from biological dark adaptation mechanisms, simulating functions like pupils and photoreceptor cells. Instead of relying on extensive machine learning with day-to-night image conversions, it focuses on image fusion and gamma correction to train deep neural networks for dark adaptation. This research also involves creating a simulated environment ranging from 0 lux to high brightness, testing the limits of object detection, and offering a high dynamic range testing method. Results indicate that the dark adaptation model developed in this study improves the mean average precision (mAP) by 1.5-6% compared to traditional models. Our model is capable of functioning in both twilight and night, showcasing academic novelty. Future developments could include using virtual light in specific image areas or integrating with smart car lighting to enhance detection accuracy, thereby improving safety for pedestrians and drivers.

IInception-CBAM-IBiGRU based fault diagnosis method for asynchronous motors

Aiming at the problems of insufficient extraction of asynchronous motor fault features by traditional deep learning algorithms and poor diagnosis of asynchronous motor faults in robust noise environments, this paper proposes an end-to-end fault diagnosis method for asynchronous motors based on IInception-CBAM-IBiGRU. The method first uses a signal-to-grayscale image conversion method to convert one-dimensional vibration signals into two-dimensional images and initially extracts shallow features through two-dimensional convolution; then the Improved Inception (IInception) module is used as a residual block to learning features at different scales with a residual structure, and extracts its important feature information through the Convolutional Block Attention Module (CBAM) to extract important feature information and adjust the weight parameters; then the feature information is input to the Improved Bi-directional Gate Recurrent Unit (IBiGRU) to extract its timing features further; finally, the fault identification is achieved by the SoftMax function. The primary hyperparameters in the model are optimized by the Weighted Mean Of Vectors Algorithm (INFO). The experimental results show that the method is effective in fault diagnosis of asynchronous motors, with an accuracy rate close to 100%, and can still maintain a high accuracy rate under the condition of low noise ratio, with good robustness and generalization ability.

From tomographic imaging to numerical simulations: an open-source workflow for true morphology mesoscale FE meshes

Full-field techniques such as tomography are becoming progressively more central in the study of complex phenomena, in particular where spatiotemporal evolution is crucial, as in moisture transport or crack initiation in porous media. These techniques provide a unique insight in the local process whose quantification allows the improvement of our understanding and of the models describing them. Nevertheless, the model validation can be pushed further by attempting to explicitly represent the heterogeneities and simulate their role in the processes. Once validated, these models can be used to perform “virtual experiments”, and overcome the limitations of the experiments (e.g., sample size and number, fine control of the boundary and initial conditions). This study proposes a connection between tomography images and mesoscale models through a workflow that mainly employs open-source tools. This workflow is illustrated through the digitization of a Portland cement concrete sample, stemming from neutron tomographies and resulting in a numerical finite element mesh. The proposed workflow is flexible, allowing for the conversion of images from various sources, such as x-ray or neutron tomographies, to different numerical representations of the domain, such as finite element meshes or even a discrete domain required by discrete element methods, while preserving real morphologies with an accuracy proportionate to the specific need of the problem. Beside its generalizability, our method also offers automated labelling of the different domains and boundaries in both the volumetric and surface meshes, which is often necessary for assigning material properties and boundary conditions. Finally, the series of image, geometry and mesh processing steps described in this work are made available on a GitHub repository.

Application of Makeup Image Optimization Recommendation System through the Analysis of BeautyGAN Based on Deep Learning

The purpose of this study was to identify the makeup preference of users and suggest a method to optimize the makeup style by using the preferred image for each age group through the analysis of BeautyGAN. Through this, you can propose a customized makeup style that suits you, and provide beneficial services to the makeup industry and consumers. In addition, by developing and validating new methods that effectively combine deep learning and vision systems, we aim to innovate makeup-related image conversion technology and contribute to academic and practical advances in this field. For this purpose, reference images suitable for each image were collected to implement image optimization for each age group, the input data reflected the researcher’s image, and the face was aligned and resized, after removing images with low resolution or poor lighting conditions. As a result of the performance evaluation of the BeautyGAN model, it was confirmed that the existing image was 51.26%, which is close to the BeautyGAN image of 38.89%. These results are judged to be able to provide customized makeup style suggestions or adjusted makeup effects that reflect the user’s preferences from an academic point of view, and from a practical point of view, it will be possible to improve the quality of customized beauty services by suggesting makeup styles that suit the characteristics of customers more accurately and quickly.

Synthesis and Properties of Bright Red-to-NIR BODIPY Dyes for Targeting Fluorescence Imaging and Near-Infrared Photothermal Conversion.

An efficient synthesis of 3-pyrrolylBODIPY dyes has been developed from a rational mixture of various aromatic aldehydes and pyrrole in a straightforward condensation reaction, followed by in situ successively oxidative nucleophilic substitution using a one-pot strategy. These resultant 3-pyrrolylBODIPYs without blocking substituents not only exhibit the finely tunable photophysical properties induced by the flexible meso-aryl substituents but also serve as a valuable synthetic framework for further selective functionalization. As a proof of such potential, one 3-pyrrolylBODIPY dye (581/603 nm) through the installation of the morpholine group is applicable for lysosome-targeting imaging. Furthermore, an ethene-bridged 3,3'-dipyrrolylBODIPY dimer was constructed, which displayed a near-infrared (NIR) emission extended to 1200 nm with a large fluorescence brightness (2840 M-1 cm-1). The corresponding dimer nanoparticles (NPs) afforded a high photothermal conversion efficiency (PCE) value of 72.5%, eventually resulting in favorable photocytotoxicity (IC50 = 9.4 μM) and efficient in vitro eradication of HeLa cells under 808 nm laser irradiation, highlighting their potential application for photothermal therapy in the NIR window.

Integrating tabular data through image conversion for enhanced diagnosis: A novel intelligent decision support system for stratifying obstructive sleep apnoea patients using convolutional neural networks.

High-dimensional databases make it difficult to apply traditional learning algorithms to biomedical applications. Recent developments in computer technology have introduced deep learning (DL) as a potential solution to these difficulties. This study presents a novel intelligent decision support system based on a novel interpretation of data formalisation from tabular data in DL techniques. Once defined, it is used to diagnose the severity of obstructive sleep apnoea, distinguishing between moderate to severe and mild/no cases. The study uses a complete database extract from electronic health records of 2472 patients, including anthropometric data, habits, medications, comorbidities, and patient-reported symptoms. The novelty of this methodology lies in the initial processing of the patients' data, which is formalised into images. These images are then used as input to train a convolutional neural network (CNN), which acts as the inference engine of the system. The initial tests of the system were performed on a set of 247 samples from the Pulmonary Department of the Álvaro Cunqueiro Hospital in Vigo (Galicia, Spain), with an AUC value of ≈ 0.8. This study demonstrates the benefits of an intelligent decision support system based on a novel data formalisation approach that allows the use of advanced DL techniques starting from tabular data. In this way, the ability of CNNs to recognise complex patterns using visual elements such as gradients and contrasts can be exploited. This approach effectively addresses the challenges of analysing large amounts of tabular data and reduces common problems such as bias and variance, resulting in improved diagnostic accuracy.

Investigating Models for the Transcription of Mathematical Formulas in Images

The automated transcription of mathematical formulas represents a complex challenge that is of great importance for digital processing and comprehensibility of mathematical content. Consequently, our goal was to analyze state-of-the-art approaches for the transcription of printed mathematical formulas on images into spoken English text. We focused on two approaches: (1) The combination of mathematical expression recognition (MER) models and natural language processing (NLP) models to convert formula images first into LaTeX code and then into text, and (2) the direct conversion of formula images into text using vision-language (VL) models. Since no dataset with printed mathematical formulas and corresponding English transcriptions existed, we created a new dataset, Formula2Text, for fine-tuning and evaluating our systems. Our best system for (1) combines the MER model LaTeX-OCR and the NLP model BART-Base, achieving a translation error rate of 36.14% compared with our reference transcriptions. In the task of converting LaTeX code to text, BART-Base, T5-Base, and FLAN-T5-Base even outperformed ChatGPT, GPT-3.5 Turbo, and GPT-4. For (2), the best VL model, TrOCR, achieves a translation error rate of 42.09%. This demonstrates that VL models, predominantly employed for classical image captioning tasks, possess significant potential for the transcription of mathematical formulas in images.

On the issue of ensuring secure access to information systems using biometric authentication based on a fuzzy image of the user’s identity and neural network transformations

Objective. In modern conditions, cryptographic personal authentication technologies are used to access information systems, based on processing biometric information and converting the user’s biometric images into his personal access code. A current area of research is the use of neural network technologies in organizing secure access to information systems. Biometric authentication tools can be classified as highly reliable only if they include cryptographic authentication mechanisms that work together with biometric authentication mechanisms through the conversion of biometric images into a unique cryptographic access code; in this case, a set of biometric identifiers of the user’s identity form his fuzzy image, which is used later during authentication. The purpose of the study is to develop an algorithm for providing secure access to information systems using biometric authentication based on a fuzzy image of the user’s identity and neural network transformations. Method. The development of the algorithm is based on the use of fuzzy logic methods and neural networks. Result. The work reveals the features of biometric identification of the user of information systems. An algorithm is proposed for providing secure access to information systems using biometric authentication based on a fuzzy image of the user’s personality and neural network transformations. Conclusions. Based on two-step authentication of the user’s identity, secure access to the information systems of registered users is implemented. A listing of the program code in Python for creating and training a neural network of a bioidentifier classifier is provided. The materials of the article are of practical value for specialists in the field of providing secure access to information systems using artificial intelligence.

User identification system based on 2D CQT spectrogram of EMG with adaptive frequency resolution adjustment.

User identification systems based on electromyogram (EMG) signals, generated inside the body in different signal patterns and exhibiting individual characteristics based on muscle development and activity, are being actively researched. However, nonlinear and abnormal signals constrain conventional user identification using EMG signals in improving accuracy by using the 1-D feature from each time and frequency domain. Therefore, multidimensional features containing time-frequency information extracted from EMG signals have attracted much attention to improving identification accuracy. We propose a user identification system using constant Q transform (CQT) based 2D features whose time-frequency resolution is customized according to EMG signals. The proposed user identification system comprises data preprocessing, CQT-based 2D image conversion, convolutional feature extraction, and classification by convolutional neural network (CNN). The experimental results showed that the accuracy of the proposed user identification system using CQT-based 2D spectrograms was 97.5%, an improvement of 15.4% and 2.1% compared to the accuracy of 1D features and short-time Fourier transform (STFT) based user identification, respectively.

Current role of radiomics and radiogenomics in predicting oncological outcomes in bladder cancer

Abstract Background Radiomics refers to the conversion of medical images into high-throughput, quantifiable data to analyze disease patterns, aid decision-making, and predict prognosis. Radiogenomics is an extension of radiomics and involves a combination of conventional radiomics techniques with molecular analysis in the form of genomic and transcriptomic data. In the field of bladder cancer, studies have investigated the development, implementation, and efficacy of radiomic and radiogenomic nomograms in predicting tumor grade, gene expression, and oncological outcomes, with variable results. We aimed to perform a systematic review of the current literature to investigate the development of a radiomics-based nomogram to predict oncological outcomes in bladder cancer. Materials and methods The Medline, EMBASE, and Web of Science databases were searched up to February 17, 2023. Gray literature was also searched to further identify other suitable publications. Quality assessment of the included studies was performed using the Quality Assessment of Diagnostic Accuracy Studies 2 and Radiomics Quality Score. Results Radiogenomic nomograms generally had good performance in predicting the primary outcome across the included studies. The median area under the curve, sensitivity, and specificity across the included studies were 0.83 (0.63–0.973), 0.813, and 0.815, respectively, in the training set and 0.75 (0.702–0.838), 0.723, and 0.652, respectively, in the validation set. Conclusions Several studies have demonstrated the predictive potential of radiomic and radiogenomic models in advanced pelvic oncology. Further large-scale studies in a prospective setting are required to further validate results and allow generalized use in modern medicine.

L2NLF: a novel linear-to-nonlinear framework for multi-modal medical image registration.

In recent years, deep learning has ushered in significant development in medical image registration, and the method of non-rigid registration using deep neural networks to generate a deformation field has higher accuracy. However, unlike monomodal medical image registration, multimodal medical image registration is a more complex and challenging task. This paper proposes a new linear-to-nonlinear framework (L2NLF) for multimodal medical image registration. The first linear stage is essentially image conversion, which can reduce the difference between two images without changing the authenticity of medical images, thus transforming multimodal registration into monomodal registration. The second nonlinear stage is essentially unsupervised deformable registration based on the deep neural network. In this paper, a brand-new registration network, CrossMorph, is designed, a deep neural network similar to the U-net structure. As the backbone of the encoder, the volume CrossFormer block can better extract local and global information. Booster promotes the reduction of more deep features and shallow features. The qualitative and quantitative experimental results on T1 and T2 data of 240 patients' brains show that L2NLF can achieve excellent registration effect in the image conversion part with very low computation, and it will not change the authenticity of the converted image at all. Compared with the current state-of-the-art registration method, CrossMorph can effectively reduce average surface distance, improve dice score, and improve the deformation field's smoothness. The proposed methods have potential value in clinical application.

Coal–Rock Data Recognition Method Based on Spectral Dimension Transform and CBAM-VIT

Coal–gangue sorting is a vital component of intelligent mine construction. As intelligent manufacturing continued to advance, data-driven coal–gangue recognition emerged as a prominent research topic. However, conventional data-driven methods for coal–gangue recognition heavily rely on expert-extracted features. The process of feature extraction is labor-intensive and significantly impacts the final outcome. Deep learning (DL) offers an effective approach to automatically extract features from raw data. Among the various DL techniques, convolutional neural networks (CNNs) have proven to be particularly effective. In this paper, we propose an intelligent method for recognizing coal–rock by fusing multiple preprocessing techniques applied to near-infrared spectra and employing dual attention. Initially, a signal-to-RGB image conversion method is applied to fuse three types of preprocessing data, namely first-order differential, second-order differential, and standard normal transform, into an RGB image representation. Subsequently, we propose a neural network model (CBAM-VIT) that integrates the convolutional block attention mechanism (CBAM) and Vision Transformer (VIT). When evaluated on the coal–rock dataset, this model achieves an accuracy of 98.5%, surpassing the performance of VIT (95.3%), VGG-16 (89%), and AlexNet (82%). The comparative results clearly demonstrate that the proposed coal–gangue recognition method yields significant improvements in classification outcomes.

SAR-to-Optical Image Translation via an Interpretable Network

Synthetic aperture radar (SAR) is prevalent in the remote sensing field but is difficult to interpret by human visual perception. Recently, SAR-to-optical (S2O) image conversion methods have provided a prospective solution. However, since there is a substantial domain difference between optical and SAR images, they suffer from low image quality and geometric distortion in the produced optical images. Motivated by the analogy between pixels during the S2O image translation and molecules in a heat field, a thermodynamics-inspired network for SAR-to-optical image translation (S2O-TDN) is proposed in this paper. Specifically, we design a third-order finite difference (TFD) residual structure in light of the TFD equation of thermodynamics, which allows us to efficiently extract inter-domain invariant features and facilitate the learning of nonlinear translation mapping. In addition, we exploit the first law of thermodynamics (FLT) to devise an FLT-guided branch that promotes the state transition of the feature values from an unstable diffusion state to a stable one, aiming to regularize the feature diffusion and preserve image structures during S2O image translation. S2O-TDN follows an explicit design principle derived from thermodynamic theory and enjoys the advantage of explainability. Experiments on the public SEN1-2 dataset show the advantages of the proposed S2O-TDN over the current methods with more delicate textures and higher quantitative results.