Diverse Images Research Articles

A novel machine learning model for breast cancer detection using mammogram images.

The most fatal disease affecting women worldwide now is breast cancer. Early detection of breast cancer enhances the likelihood of a full recovery and lowers mortality. Based on medical imaging, researchers from all around the world are developing breast cancer screening technologies. Due to their rapid progress, deep learning algorithms have caught the interest of many in the field of medical imaging. This research proposes a novel method in mammogram image feature extraction with classification and optimization using machine learning in breast cancer detection. The input image has been processed for noise removal, smoothening, and normalization. The input image features were extracted using probabilistic principal component analysis for detecting the presence of tumors in mammogram images. The extracted tumor region is classified using the Naïve Bayes classifier and transfer integrated convolution neural networks. The classified output has been optimized using firefly binary grey optimization and metaheuristic moth flame lion optimization. The experimental analysis has been carried out in terms of different parameters based on datasets. The proposed framework used an ensemble model for breast cancer that made use of the proposed Bayes + FBGO and TCNN + MMFLO classifier and optimizer for diverse mammography image datasets. The INbreast dataset was evaluated using the proposed Bayes + FBGO and TCNN + MMFLO classifiers, which achieved 95% and 98% accuracy, respectively.

Incremental-based YoloV3 model with hyper-parameter optimization for product image classification in E-commerce sector

Over the past few years, the E-commerce industry has grown tremendously for selling products to consumers. Here, the consumer can easily purchase the products from their residing seats and gets the products at the door step. Also, the image quality is suggested and also the image classification is intruded to meet the consumer prerequisite. Most of the image classification models are developed by machine learning approaches to improve the reliability, scalability, and accuracy level. Due to the heterogeneous nature of images, it restricts the features and lengthy dimensions for datasets which it degrades the performance whereas the classification of products becomes a cumbersome situation. In order to resolve the problem raised by existing methods, a novel product image classification model is proposed for the fashion E-commerce sector. The prime aspect of the proposed model is to build an effective E-commerce sector for a layman, where every individual can purchase the products appropriately. Initially, the dataset is constructed with diverse product images that are obtained from different E-commerce fields with various classes. Consequently, a novel Incremental-based Improved YoloV3 with Hyper-parameter Optimization (II-Yolov3-HO) model is introduced to enhance the classification performance. In the proposed model, some of the hyper parameters like convolution and pooling layers are replaced with the Visual Geometry Group16 (VGG16) model, the remaining convolutional layers are enhanced by the atrous convolution operation, and pooling layers are modified with Atrous Spatial Pyramid Pooling (ASPP). In entire architecture, the weight factor is optimized using Predator-based Squirrel Search Algorithm (PDP-SSA) algorithm. Thus, the new E-commerce sector becomes free to categorize the products, and if it deals with a new product, it is declared as unknown data, which is then trained again in the proposed model. Hence, the concept of classification score matching with the threshold value is accomplished to deal the untrained product and the training weight is updated optimally using the proposed PDP-SSA. Finally, the enhanced model is compared over the state-of-the-art classification, provides the expected result for significantly classifying the multiple classes of products. The developed PDP-SSA-II-YoloV3-HO model is also able to perform recommendations in E-commerce sites even when the input is given as images other than text for searching the items. These uploaded images are analyzed in the developed II-Yolov3-HO model for providing the relevant items to the customer based on their features. The developed PDP-SSA-II-YoloV3-HO model achieves 99.46 % regarding accuracy. Throughout the validation, the developed model outperforms enhanced and reliable performance than the existing approaches.

A Collaborative Self-supervised Domain Adaptation for Low-Quality Medical Image Enhancement.

Medical image analysis techniques have been employed in diagnosing and screening clinical diseases. However, both poor medical image quality and illumination style inconsistency increase uncertainty in clinical decision-making, potentially resulting in clinician misdiagnosis. The majority of current image enhancement methods primarily concentrate on enhancing medical image quality by leveraging high-quality reference images, which are challenging to collect in clinical applications. In this study, we address image quality enhancement within a fully self-supervised learning setting, wherein neither high-quality images nor paired images are required. To achieve this goal, we investigate the potential of self-supervised learning combined with domain adaptation to enhance the quality of medical images without the guidance of high-quality medical images. We design a Domain Adaptation Self-supervised Quality Enhancement framework, called DASQE. More specifically, we establish multiple domains at the patch level through a designed rule-based quality assessment scheme and style clustering. To achieve image quality enhancement and maintain style consistency, we formulate the image quality enhancement as a collaborative self-supervised domain adaptation task for disentangling the low-quality factors, medical image content, and illumination style characteristics by exploring intrinsic supervision in the low-quality medical images. Finally, we perform extensive experiments on six benchmark datasets of medical images, and the experimental results demonstrate that DASQE attains state-of-the-art performance. Furthermore, we explore the impact of the proposed method on various clinical tasks, such as retinal fundus vessel/lesion segmentation, nerve fiber segmentation, polyp segmentation, skin lesion segmentation, and disease classification. The results demonstrate that DASQE is advantageous for diverse downstream image analysis tasks.

Black and White Image Colorization with Deep Learning

Abstract: The project "Black and White Image Colorization with Deep Learning" develops a system to automatically infuse colors into grayscale images through cutting-edge deep learning approaches. Utilizing convolutional neural networks (CNNs) and generative adversarial networks (GANs), the system aims to deliver both realistic and visually pleasing colorization outcomes. This process includes training the deep learning models on diverse image datasets, optimizing the model parameters, and using advanced loss functions to ensure precise color representation.This project addresses the complex challenges associated with adding colors to monochrome images and also seeks to improve user experience by introducing customizable options such as style modification and color tweaks. By merging deep learning with computer vision techniques, this initiative is poised to contribute significantly to various image processing fields, especially in the restoration of historical images and the enhancement of visual content production. With thorough experimentation and ongoing refinements, the project aspires to establish a reliable and easy-to-use system that advances the state of image colorization technology in the modern digital age..

Unveiling Anomalies in Surveillance Videos

Abstract: The implementation of Convolutional Neural Network (CNN) algorithm for the detection of anomalies like fire and potholes in images or videos. Leveraging deep learning technique, the model aims to accurately identify fire, potholes, etc. within various contexts. The CNN architecture is trained on a dataset comprising diverse images to enhance its ability to generalize. This proposed project not only addresses safety concerns by promoting helmet usage and proper maintenance of roads but also contributes to the broader field of computer vision and object detection. This implementation involves training the CNN on annotated datasets, fine-tuning the model for optimal performance, and integrating it into a practical application for efficient anomaly identification. With a focus on real-time detection, the system utilizes image processing techniques and a trained CNN model to analyze visual data from cameras or video feeds. The system's versatility allows integration into surveillance systems, industrial sites, or any environment where fire detection is crucial for safety. By automating the detection process, the project contributes to minimizing human error and ensuring consistent monitoring. The proposed solution holds the potential to significantly impact safety protocols, particularly in industries where protective headgear is paramount. Potholes pose a significant threat to both drivers and pedestrians, leading to accidents and infrastructure damage. The proposed solution leverages CNN's ability to effectively process visual data, making it well-suited for image recognition tasks. Our approach involves training the CNN on a diverse dataset of road images to enable it to accurately identify potholes. The model will learn distinctive features and patterns associated with potholes, allowing for robust detection under various environmental conditions. Real-time implementation on embedded systems or cameras along roadways will enable instantaneous identification and alerting.

DG2GAN: improving defect recognition performance with generated defect image sample

This article aims to improve the deep-learning-based surface defect recognition. In actual manufacturing processes, there are issues such as data imbalance, insufficient diversity, and poor quality of augmented data in the collected image data for product defect recognition. A novel defect generation method with multiple loss functions, DG2GAN is presented in this paper. This method employs cycle consistency loss to generate defect images from a large number of defect-free images, overcoming the issue of imbalanced original training data. DJS optimized discriminator loss is introduced in the added discriminator to encourage the generation of diverse defect images. Furthermore, to maintain diversity in generated images while improving image quality, a new DG2 adversarial loss is proposed with the aim of generating high-quality and diverse images. The experiments demonstrated that DG2GAN produces defect images of higher quality and greater diversity compared with other advanced generation methods. Using the DG2GAN method to augment defect data in the CrackForest and MVTec datasets, the defect recognition accuracy increased from 86.9 to 94.6%, and the precision improved from 59.8 to 80.2%. The experimental results show that using the proposed defect generation method can obtain sample images with high quality and diversity and employ this method for data augmentation significantly enhances surface defect recognition technology.

Тестування методу топографічної спорідненості зображень на зображеннях поверхні землі

Introduction. In connection with the development of the method of topographic affinity of images, it became necessary to conduct its testing according to the criteria of workability and efficiency. The purpose of the paper is testing of the method of determining the topographic affinity of images based on taking into account the detected special zones on the images of the natural landscape for the autonomous navigation of UAVs. Results. According to the results of testing for three tasks, the method showed its effectiveness at the level of 100%. Conclusions. The method of topographic affinity of images can work with a large number of complex and diverse images of the Earth's surface, which cannot be analyzed by other known methods, and with high efficiency. It can be used to build a system of autonomous navigation of UAVs separately or together with other methods. Keywords: unmanned aerial vehicle, unmanned aircraft complex, autonomous navigation, special points, special areas, method of analysis of special areas of images.

Machine Learning Assisted Prediction of Porosity and Related Properties Using Digital Rock Images.

Accurately estimating reservoir rock properties is paramount for modeling the storage and flow of fluids (hydrocarbon, carbon dioxide, and groundwater) in porous media. However, existing laboratory techniques to measure rock properties are usually time-consuming, expensive, and computationally intensive. This work proposes an efficient workflow that uses the machine learning algorithm, based on the convolutional neural network (CNN) framework, to predict rock properties from microcomputed tomography (micro-CT) X-ray images. The workflow involves data preprocessing, label extraction, training, and prediction using the segmented images of the rock to predict porosity, throat area, and pore surface area, which are essential for pore-scale modeling. The model was trained and validated on the Bentheimer sandstone, which was then used to predict properties of other sandstones (Castlegate and Leopard) with different pore structures and flow properties. The model yielded a good prediction for the throat and pore surface area but a significant error for porosity. Subsequently, a new complex model was trained and validated using diverse images from Bentheimer and an additional rock Castlegate, which was then used to predict the properties of Leopard sandstone. The new model improved the prediction of each property, resulting in mean absolute percentage error (MAPE) values of 2.19%, 3.04%, and 6.08% for porosity, pore surface area, and throat area with the binary images, respectively. In addition, we present a novel data-driven method using a simple regression model to predict the absolute permeability of a digital rock sample using the pore network parameters as predictors. The extreme gradient boost (XGBoost), which performed the best among several machine algorithms, was trained and validated using digital rock images from Bentheimer and Castlegate sandstone. The generated model was then used to predict the absolute permeability of the Leopard sandstone with an R 2 of 0.813, which was a significant improvement over the model generated solely by using either the Bentheimer or the Castlegate sandstone images. Furthermore, our analysis showed that the tortuosity had the most significant effect on the absolute permeability prediction of the rock sample. This study showed that we can reliably predict the morphological properties of porous media using computationally efficient models generated from digital rock images, which can be used to build a regression model to predict the crucial petrophysical properties needed to model the flow of fluids in porous media.

TOMGPT: Reliable Text-Only Training Approach for Cost-Effective Multi-modal Large Language Model

Multi-modal large language models (MLLMs), such as GPT-4, exhibit great comprehension capabilities on human instruction, as well as zero-shot ability on new downstream multi-modal tasks. To integrate the different modalities within a unified embedding space, previous MLLMs attempted to conduct visual instruction tuning with massive and high-quality image-text pair data, which requires substantial costs in data collection and training resources. In this article, we propose TOMGPT (Text-Only training Multi-modal GPT), a cost-effective MLLM tuned solely on easily accessible text data with much fewer resources. Along with pre-trained visual-linguistic coupled modality space (e.g., CLIP and ALIGN model), a text-only training strategy is devised to further project the aligned multi-modal latent space to that of LLM, endowing the LLM with visual comprehension capabilities in an efficient manner. Instead of enormous image-text training data required by previous MLLMs, we find that TOMGPT can be well-tuned with fewer yet diverse GPT-generated free-form text data, as we establish the semantic connection between LLM and pre-trained vision-language model. A quantitative evaluation is conducted on both MME and LVLM, which are recently released and extensively utilized MLLM benchmarks. The experiments reveal that TOMGPT achieved reliable performance compared to numerous models trained on a large amount of image-text pair data. Case studies are also presented, demonstrating TOMGPT’s broad understanding and dialogue capabilities across diverse image categories.

PATH-13. LEARNED RESIZING WITH EFFICIENT TRAINING (LRET) FACILITATES IMPROVED PERFORMANCE OF LARGE-SCALE BRAIN TUMOR HISTOLOGY IMAGE CLASSIFICATION MODELS

Abstract BACKGROUND Histologic examination is vital in oncology research and diagnostics. The adoption of digital scanning of whole slide images (WSI) has created an opportunity to leverage deep learning-based image classification methods to enhance diagnosis and risk stratification. However, technical limitations prevent training and deployment of accurate comprehensive multiclass deep convolutional neural networks (DCNN) models for histopathology image classification. The input dimensions of DCNN architectures are small compared to the typical pathologist field of view, degrading performance by excluding important architectural features. Furthermore, data requirements for comprehensive models are sufficiently large to overwhelm the system memory during training. METHODS A method termed Learned Resizing with Efficient Training (LRET) was developed to address the main limitations of traditional histopathology classification model training. The LRET method couples efficient training techniques with image resizing to facilitate seamless integration of larger histology image patches into state-of-the-art classification models while preserving important structural information. The LRET method was coupled with two distinct resizing techniques to train three diverse histology image datasets using five different DCNN architectures. Performance metrics were compared on cross validation and hold out test sets. RESULTS LRET-trained models were flexible to multiple input patch dimensions and DCNN models. We demonstrated performance improvement across all datasets while significantly reducing the training time and resources over traditional methods. Using a large-scale, multiclass brain tumor classification dataset consisting of 74 distinct histopathologic classes, LRET-trained models outperformed existing methods by 15-28% in accuracy, yielding 94% accuracy for the best model. CONCLUSION The LRET method for DCNN training significantly enhances the performance of large-scale multiclass histopathology image classification. The implications of this work extend to broader applications within medical imaging and beyond, where efficient integration of high-resolution images into deep learning pipelines is paramount for driving advancements research and clinical practice.

Universal Image Restoration with Text Prompt Diffusion.

Universal image restoration (UIR) aims to accurately restore images with a variety of unknown degradation types and levels. Existing methods, including both learning-based and prior-based approaches, heavily rely on low-quality image features. However, it is challenging to extract degradation information from diverse low-quality images, which limits model performance. Furthermore, UIR necessitates the recovery of images with diverse and complex types of degradation. Inaccurate estimations further decrease restoration performance, resulting in suboptimal recovery outcomes. To enhance UIR performance, a viable approach is to introduce additional priors. The current UIR methods have problems such as poor enhancement effect and low universality. To address this issue, we propose an effective framework based on a diffusion model (DM) for universal image restoration, dubbed ETDiffIR. Inspired by the remarkable performance of text prompts in the field of image generation, we employ text prompts to improve the restoration of degraded images. This framework utilizes a text prompt corresponding to the low-quality image to assist the diffusion model in restoring the image. Specifically, a novel text-image fusion block is proposed by combining the CLIP text encoder and the DA-CLIP image controller, which integrates text prompt encoding and degradation type encoding into time step encoding. Moreover, to reduce the computational cost of the denoising UNet in the diffusion model, we develop an efficient restoration U-shaped network (ERUNet) to achieve favorable noise prediction performance via depthwise convolution and pointwise convolution. We evaluate the proposed method on image dehazing, deraining, and denoising tasks. The experimental results indicate the superiority of our proposed algorithm.

Leukemia Detection and Classification Based on Machine Learning and CNN: A Review

Advancements in data mining methods have significantly improved disease diagnosis, particularly in the realm of leukemia detection. Leukemia, a complex cancer affecting white blood cells, poses significant challenges in diagnosis and management due to its diverse manifestations. Various machine learning algorithms, including Convolutional Neural Networks (CNNs), Support Vector Machines (SVMs), Random Forests (RF), Decision Trees (DTs), K-Nearest Neighbors (K-NN), Logistic regression (LR) and Naïve Bayes (NB) classifiers, have been employed to accurately classify leukemia cases based on diverse datasets and image analyses. This paper provides a comprehensive overview and comparison of these classification techniques, highlighting their effectiveness in diagnosing different leukemia subtypes. Additionally, the paper discusses the methodology and findings of several studies focusing on leukemia detection, emphasizing the significance of machine learning in enhancing diagnostic accuracy and treatment planning. Furthermore, it explores the challenges and future directions in leveraging machine learning for leukemia diagnosis, including the need for standardized datasets, algorithm refinement, and integration with clinical data for personalized treatment strategies.

DiffDD: A surface defect detection framework with diffusion probabilistic model

Automated defect detection is critical to improve product quality, reduce costs, and enhance productivity in the manufacturing process. More recently, diffusion-based generative models have entered a new realm of vision tasks by offering high-quality and diverse images. However, the diffusion-based defect detection model fails to be researched, and generative models typically require postprocessing modules to achieve the segmentation of defective regions. Furthermore, generic diffusion models struggle to reconstruct defective regions while preserving the semantic information of the inspection images. To address these issues, we propose a Diffusion-based Defect Detection (DiffDD) framework, comprising a pre-trained backbone (PvTv2) and diffusion probabilistic model (DPM), for surface defect detection. The backbone is employed to extract multi-scale feature maps from input images supervised by pixel-wise labels. These pyramid maps interact with DPM to guide the diffusion process, refining the semantic information of the reconstructed image. Additionally, a semantic restoration (SR) module is used to restore the output of the backbone for loss computation with the pixel-wise label. A conditional image prior method is implemented to constrain the randomness of DPM, aiding the model in generating segmentation masks corresponding to the inspection images. Experimental results on NEU_SEG and MT demonstrate that our model outperforms state-of-the-art (SOTA) methods by a significant margin, indicating the generalization and effectiveness of the proposed DiffDD framework.

High Fidelity Makeup via 2D and 3D Identity Preservation Net

In this article, we address the challenging makeup transfer task, aiming to transfer makeup from a reference image to a source image while preserving facial geometry and background consistency. Existing deep neural network-based methods have shown promising results in aligning facial parts and transferring makeup textures. However, they often neglect the facial geometry of the source image, leading to two adverse effects: (1) alterations in geometrically relevant facial features, causing face flattening and loss of personality, and (2) difficulties in maintaining background consistency, as networks cannot clearly determine the face-background boundary. To jointly tackle these issues, we propose the High Fidelity Makeup via two-dimensional (2D) and 3D Identity Preservation Network (IP23-Net), to the best of our knowledge, a novel framework that leverages facial geometry information to generate more realistic results. Our method comprises a 3D Shape Identity Encoder, which extracts identity and 3D shape features. We incorporate a 3D face reconstruction model to ensure the three-dimensional effect of face makeup, thereby preserving the characters’ depth and natural appearance. To preserve background consistency, our Background Correction Decoder automatically predicts an adaptive mask for the source image, distinguishing the foreground and background. In addition to popular benchmarks, we introduce a new large-scale High Resolution Synthetic Makeup Dataset containing 335,230 diverse high-resolution face images to evaluate our method’s generalization ability. Experiments demonstrate that IP23-Net achieves high-fidelity makeup transfer while effectively preserving background consistency. The code will be made publicly available.

A transfer learning enabled approach for ocular disease detection and classification.

Ocular diseases pose significant challenges in timely diagnosis and effective treatment. Deep learning has emerged as a promising technique in medical image analysis, offering potential solutions for accurately detecting and classifying ocular diseases. In this research, we propose Ocular Net, a novel deep learning model for detecting and classifying ocular diseases, including Cataracts, Diabetic, Uveitis, and Glaucoma, using a large dataset of ocular images. The study utilized an image dataset comprising 6200 images of both eyes of patients. Specifically, 70% of these images (4000 images) were allocated for model training, while the remaining 30% (2200 images) were designated for testing purposes. The dataset contains images of five categories that include four diseases, and one normal category. The proposed model uses transfer learning, average pooling layers, Clipped Relu, Leaky Relu and various other layers to accurately detect the ocular diseases from images. Our approach involves training a novel Ocular Net model on diverse ocular images and evaluating its accuracy and performance metrics for disease detection. We also employ data augmentation techniques to improve model performance and mitigate overfitting. The proposed model is tested on different training and testing ratios with varied parameters. Additionally, we compare the performance of the Ocular Net with previous methods based on various evaluation parameters, assessing its potential for enhancing the accuracy and efficiency of ocular disease diagnosis. The results demonstrate that Ocular Net achieves 98.89% accuracy and 0.12% loss value in detecting and classifying ocular diseases by outperforming existing methods.

Cellpose as a reliable method for single-cell segmentation of autofluorescence microscopy images.

Autofluorescence microscopy uses intrinsic sources of molecular contrast to provide cellular-level information without extrinsic labels. However, traditional cell segmentation tools are often optimized for high signal-to-noise ratio (SNR) images, such as fluorescently labeled cells, and unsurprisingly perform poorly on low SNR autofluorescence images. Therefore, new cell segmentation tools are needed for autofluorescence microscopy. Cellpose is a deep learning network that is generalizable across diverse cell microscopy images and automatically segments single cells to improve throughput and reduce inter-human biases. This study aims to validate Cellpose for autofluorescence imaging, specifically from multiphoton intensity images of NAD(P)H. Manually segmented nuclear masks of NAD(P)H images were used to train new Cellpose models. These models were applied to PANC-1 cells treated with metabolic inhibitors and patient-derived cancer organoids (across 9 patients) treated with chemotherapies. These datasets include co-registered fluorescence lifetime imaging microscopy (FLIM) of NAD(P)H and FAD, so fluorescence decay parameters and the optical redox ratio (ORR) were compared between masks generated by the new Cellpose model and manual segmentation. The Dice score between repeated manually segmented masks was significantly lower than that of repeated Cellpose masks (p<0.0001) indicating greater reproducibility between Cellpose masks. There was also a high correlation (R 2 >0.9) between Cellpose and manually segmented masks for the ORR, mean NAD(P)H lifetime, and mean FAD lifetime across 2D and 3D cell culture treatment conditions. Masks generated from Cellpose and manual segmentation also maintain similar means, variances, and effect sizes between treatments for the ORR and FLIM parameters. Overall, Cellpose provides a fast, reliable, reproducible, and accurate method to segment single cells in autofluorescence microscopy images such that functional changes in cells are accurately captured in both 2D and 3D culture.

An electric bicycle tracking algorithm for improved traffic management

This paper proposes an efficient electric bicycle tracking algorithm, EBTrack, utilizing the high-precision and lightweight YOLOv7 as the target detector to enhance the efficiency of illegal detection and recognition of electric bicycles. The EBTrack effectively captures the position and trajectory of electric bicycles in complex traffic monitoring scenarios. Firstly, we introduce the feature extraction network, ResNetEB, specifically designed for feature re-identification of electric bicycles. To maintain real-time performance, feature extraction is performed only when generating new object IDs, minimizing the impact on processing speed. Secondly, for accurate target trajectory prediction, we incorporate an adaptive modulated noise scale Kalman filter. Additionally, considering the uncertainty of electric bicycle entry directions in traffic monitoring scenarios, we design a specialized matching mechanism to reduce frequent ID switching. Finally, to validate the algorithm's effectiveness, we have collected diverse video image data of electric bicycle and urban road traffic in Hefei, Anhui Province, encompassing different perspectives, time periods, and weather conditions. We have trained the proposed detector and have evaluated its tracking performance on this comprehensive dataset. Experimental results demonstrate that EBTrack achieves impressive accuracy, with 89.8 % MOTA (Multiple Object Tracking Accuracy) and 94.2 % IDF1 (ID F1-Score). Furthermore, the algorithm effectively reduces ID switching, significantly improving tracking stability and continuity.

MetaV: A Pioneer in feature Augmented Meta-Learning Based Vision Transformer for Medical Image Classification.

Image classification, a fundamental task in computer vision, faces challenges concerning limited data handling, interpretability, improved feature representation, efficiency across diverse image types, and processing noisy data. Conventional architectural approaches have made insufficient progress in addressing these challenges, necessitating architectures capable of fine-grained classification, enhanced accuracy, and superior generalization. Among these, the vision transformer emerges as a noteworthy computer vision architecture. However, its reliance on substantial data for training poses a drawback due to its complexity and high data requirements. To surmount these challenges, this paper proposes an innovative approach, MetaV, integrating meta-learning into a vision transformer for medical image classification. N-way K-shot learning is employed to train the model, drawing inspiration from human learning mechanisms utilizing past knowledge. Additionally, deformational convolution and patch merging techniques are incorporated into the vision transformer model to mitigate complexity and overfitting while enhancing feature representation. Augmentation methods such as perturbation and Grid Mask are introduced to address the scarcity and noise in medical images, particularly for rare diseases. The proposed model is evaluated using diverse datasets including Break His, ISIC 2019, SIPaKMed, and STARE. The achieved performance accuracies of 89.89%, 87.33%, 94.55%, and 80.22% for Break His, ISIC 2019, SIPaKMed, and STARE, respectively, present evidence validating the superior performance of the proposed model in comparison to conventional models, setting a new benchmark for meta-vision image classification models.

MISP-Fuse: A progressive fusion network guided by Multi-Information supervision

The fusion of infrared and visible images aims to synthesize an image that combines the advantageous characteristics of both image sources. Nevertheless, existing image fusion algorithms often struggle to extract relevant feature information from the source images and find it challenging to balance the significance of different modality information within the source images. This leads to suboptimal fusion results and inadequately serve advanced downstream visual tasks. In order to confront this challenge, a novel image fusion algorithm, denoted as Multiple Information Supervised Progressive Fusion Network (MISP-Fuse), has been proposed. Specifically, MISP-Fuse employs an innovative and robust multi-stage encoder-decoder network called Full Scale Feature Residual network (FSFR) to extract spatial context and detailed feature information corresponding to infrared and visible images. Within this encoder-decoder network, source image feature information is progressively extracted at different stages, which are recombined at multiple scales and distilled for crucial information. Finally, the fusion image is generated through a spatial localization module named as Spatial Localization Network (SLNet). MISP-Fuse incorporates a multi-information supervision mechanism to establish a linkage between different modality information in the infrared and visible source images. It ensures that the resulting fused image not only aligns with human visual perception but also effectively serves advanced downstream visual tasks. The comparative experiments utilizing diverse image fusion benchmark datasets has been conducted. In comparison to other algorithms, MISP-Fuse demonstrated significant enhancements in comprehensive image fusion metrics, including Average Gradient (AG), Sum of Correlated Differences (SCD), and Correlation Coefficient (CC).

A single-image GAN model using self-attention mechanism and DenseNets

Image generation from a single natural image using generative adversarial networks (GANs) has attracted extensive attention recently due to the GANs’ practical ability to produce photo-realistic images and their potential applications in computer vision. However, learning a powerful generative model that generates realistic, high-quality images from only a single natural image is still a challenging problem. Training GANs in limited data regimes often causes some issues, such as overfitting, memorization, training divergence, poor image quality, and a long training time. In this study, we investigated the state-of-the-art GAN models in computer vision tasks. We conducted several experiments to deeply understand the challenges of learning a powerful generative model. We introduced a novel unconditional GAN model that produces realistic, high-quality, diverse images based on a single training image. In our model, we employed a self-attention mechanism (SAM), a densely connected convolutional network (DenseNet) architecture, and a relativistic average least-squares GAN with gradient penalty (RaLSGAN-GP) for both the generator and discriminator networks to perform image generation tasks better. SAM controls the global contextual information level. It is complementary to convolutions for large feature maps and gives the generator and discriminator more capability to capture long-range dependencies in feature maps. It compensates for the long training time and low image quality issues. DenseNet connects each layer to every other layer in a feed-forward manner to ensure maximum information flow between layers in the network. It is highly parameter efficient and requires less computation to achieve high performance. It provides improved information and gradient flow throughout the network for easy training. It has a regularizing effect that reduces overfitting in image generation. RaLSGAN-GP further improves data generation quality and the stability of our model at no computational cost and provides much more stable training. Thanks to the appropriate combination of SAM, DenseNet, and RaLSGAN-GP, our model successfully generates realistic, high-quality, diverse images while maintaining the global context of the training image. We conducted experiments, user studies, and model evaluation methods to test our model’s performance and compared it with the previous well-known models on three datasets (Places, LSUN, ImageNet). We demonstrated our model’s capability in image synthesis and image manipulation tasks. In our experiments, we showed that our model utilized parameters more efficiently, prevented overfitting, better captured the internal patch statistics of images with complex structures and textures, achieved comparable performance in single image generation tasks, and had much better visual results than its competitive peers. User studies confirmed that the images generated by our model were commonly confused with the original images. Our model can provide a powerful tool for various image manipulation tasks as well as data augmentation in domains dealing with limited training data.