Benchmark Image Research Articles

Improving Prostate Image Segmentation Based on Equilibrium Optimizer and Cross-Entropy

Over the past decade, the development of computer-aided detection tools for medical image analysis has seen significant advancements. However, tasks such as the automatic differentiation of tissues or regions in medical images remain challenging. Magnetic resonance imaging (MRI) has proven valuable for early diagnosis, particularly in conditions like prostate cancer, yet it often struggles to produce high-resolution images with clearly defined boundaries. In this article, we propose a novel segmentation approach based on minimum cross-entropy thresholding using the equilibrium optimizer (MCE-EO) to enhance the visual differentiation of tissues in prostate MRI scans. To validate our method, we conducted two experiments. The first evaluated the overall performance of MCE-EO using standard grayscale benchmark images, while the second focused on a set of transaxial-cut prostate MRI scans. MCE-EO’s performance was compared against six stochastic optimization techniques. Statistical analysis of the results demonstrates that MCE-EO offers superior performance for prostate MRI segmentation, providing a more effective tool for distinguishing between various tissue types.

Self-supervised learning from images: No negative pairs, no cluster-balancing

Learning with self-derived targets provides a non-contrastive method for unsupervised image representation learning, where the variety in targets is crucial. Recent work has achieved good performance by learning with targets obtained via cluster-balancing. However, the equal-cluster-size constraint becomes too restrictive for handling data with imbalanced categories or coming in small batches. In this paper, we propose a new clustering-based approach for non-contrastive image representation learning with no need for a particular architecture design or extra memory bank and no explicit constraints on cluster size. A key formulation is to learn embedding consistency and variable decorrelation in the cluster space by tweaking the batch-wise cross-correlation matrix towards an identity one. With this identitization loss incorporated, predicted cluster assignments of two randomly augmented views of the same image serve as targets for each other. We carried out comprehensive experimental studies of linear classification with learned representations of benchmark image datasets. Our results show that the proposed approach significantly outperforms state-of-the-art approaches and is more robust to class imbalance than those with cluster balancing.

Contrastive clustering based on generalized bias-variance decomposition

Contrastive learning demonstrates remarkable generalization performance but lacks theoretical understanding, while contrastive clustering achieves promising performance but exhibits some shortcomings. We first introduce a generalized bias-variance decomposition to study contrastive learning, then present the concept of the conformal field, which unifies instance-level contrastive loss and cluster-level de-redundancy loss (Barlow Twins). Finally, we integrate the conformal field and self-labeling to propose the outstanding contrastive clustering model D3CF. D3CF consists of two novel stages: the pre-training stage simultaneously performs instance-level contrastive learning and multi-view cluster-level redundancy reduction, bringing positive samples together and separating negative samples in the row and column space of the augmented feature matrix; to alleviate the adverse effects caused by false-negative pairs and misclustered assignments in the pre-training stage, the boosting stage enhances contrastive learning from single-positive pairs to multiple-positive pairs by leveraging cross-sample similarities, while utilizing pseudo-labels with high confidence criteria for self-labeling to correct clustering assignments. Extensive experiments on six image benchmark datasets and two text benchmarks demonstrate D3CF’s superior performance and validate the effectiveness of its components. Particularly on CIFAR-10, ImageNet-10, and STL-10, D3CF achieves average accuracies of 89.5%, 97%, and 91%, improving NMI by 5.2%, 4.8%, and 2.1%, and ARI by 7%, 7.3%, and 7.3% over the closest baseline.

Class-Level Logit Perturbation.

Features, logits, and labels are the three primary data when a sample passes through a deep neural network (DNN). Feature perturbation and label perturbation receive increasing attention in recent years. They have been proven to be useful in various deep learning approaches. For example, (adversarial) feature perturbation can improve the robustness or even generalization capability of learned models. However, limited studies have explicitly explored for the perturbation of logit vectors. This work discusses several existing methods related to class-level logit perturbation. A unified viewpoint between regular/irregular data augmentation and loss variations incurred by logit perturbation is established. A theoretical analysis is provided to illuminate why class-level logit perturbation is useful. Accordingly, new methodologies are proposed to explicitly learn to perturb logits for both the single-label and multilabel classification tasks. Meta-learning is also leveraged to determine the regular or irregular augmentation for each class. Extensive experiments on benchmark image classification datasets and their long-tail versions indicated the competitive performance of our learning method. As it only perturbs on logit, it can be used as a plug-in to fuse with any existing classification algorithms. All the codes are available at https://github.com/limengyang1992/lpl.

Identifying Truck Activity from Global Positioning System Data Based on Satellite Imagery

Road freight transportation plays a crucial role in fostering economic growth and enhancing logistics efficiency. Existing research primarily focuses on analyzing freight truck activity characteristics based on global positioning system (GPS) data of freight truck trajectories. Those studies proposed a variety of identification methods from different perspectives, but they mainly rely on rule-based methods and subjectively defined key threshold parameters, ignoring the complexity of truck freight activities and relying on long sequences of trajectory data. To bridge this gap, we propose the satellite image augmented truck activity recognition method, a structured framework that utilizes satellite imagery to identify truck activities from GPS data. Firstly, we categorize truck activities into three types: loading/unloading, resting, and driving. We extract geographic entities from the area of interest data to create the satellite image dataset of truck activity. Secondly, we evaluate the recognition accuracy of benchmark image classification models on our dataset, and ResNet50 achieves 98.04% accuracy on the test set. Thirdly, we apply the image classification model to identify truck activity from GPS trajectory data and use the recognition results of neighboring points to improve the identification results. The framework delineated in this paper entails the extraction of truck activities from GPS data at the satellite image level. Our approach holds promise in facilitating logistics operation fleets and traffic management authorities to maintain near real-time surveillance of the truck status, thereby fostering enhancements in both the efficiency and safety of freight transport.

Information hiding using approximate POSIT representation

AbstractPOSIT is a numerical system consists of sign, regime, exponential, and fraction bits to overcome some limitations of the IEEE‐754 floating point (FP) representation. This work proposes a technique to hide critical information in the least significant bits (LSBs) of the POSIT FP representation by exploiting approximate computing (AC). The proposed technique, called information hiding using POSIT and LSB (IHUPL), explores the opportunities offered by both the POSIT representation and the AC to hide information with a minimum loss in accuracy and other performance metrics. IHUPL offers two options for hiding information: either by embedding one digit of each character into each pixel or by embedding all the digits of the character at the same time into each pixel of the alpha‐red, green, blue, or black/white image. Experimental results are evaluated for benchmark images and showed that IHUPL enhanced the accuracy of embedding data into LSB of POSIT FP by an average of 5%, the image quality improvement rates of IHUPL are 19% and 16% for options 1 and 2, respectively. Besides the encoding method using IHUPL, the paper outlines an extraction decoding technique that saves the original replacement bits in a key‐image to recover the hidden security message.

A Clustering Pruning Method Based on Multidimensional Channel Information

Pruning convolutional neural networks offers a promising solution to mitigate the computational complexity challenges encountered during application deployment. However, prevalent pruning techniques primarily concentrate on model parameters or feature mapping analysis to devise static pruning strategies, often overlooking the underlying feature extraction capacity of convolutional kernels. To address this, the study first quantitatively expresses the feature extraction capability of convolutional channels from three aspects: global features, distribution metrics, and directional metrics. It explores the multi-dimensional information of the channels, calculates the overall expectation, variance, and cosine distance from the unit vector as the quantitative results of the channels. Subsequently, a clustering algorithm is employed to categorize the multidimensional information. This approach ensures that convolutional channels grouped within each cluster possess similar feature extraction capabilities. An enhanced differential evolutionary algorithm is utilized to optimize the number of clustering centers across all convolutional layers, ensuring optimal grouping. The final step involves achieving channel sparsification through the calculation of crowding distances for each sample within its designated cluster. This preserves a diverse subset of channels that are critical for maintaining model accuracy. Extensive empirical evaluations conducted on three benchmark image classification datasets demonstrate the efficacy of this method. For instance, on the ImageNet dataset, the ResNet-50 model experiences a substantial reduction in FLOPs by 58.43% while incurring a minimal decrease in TOP-1 accuracy of only 1.15%.

Computing marginal and conditional divergences between decomposable models with applications in quantum computing and earth observation

AbstractThe ability to compute the exact divergence between two high-dimensional distributions is useful in many applications, but doing so naively is intractable. Computing the $$\alpha \beta $$ α β -divergence—a family of divergences that includes the Kullback–Leibler divergence and Hellinger distance—between the joint distribution of two decomposable models, i.e., chordal Markov networks, can be done in time exponential in the treewidth of these models. Extending this result, we propose an approach to compute the exact $$\alpha \beta $$ α β -divergence between any marginal or conditional distribution of two decomposable models. In order to do so tractably, we provide a decomposition over the marginal and conditional distributions of decomposable models. We then show how our method can be used to analyze distributional changes by first applying it to the benchmark image dataset QMNIST and a dataset containing observations from various areas at the Roosevelt Nation Forest and their cover type. Finally, based on our framework, we propose a novel way to quantify the error in contemporary superconducting quantum computers.

Swin-chart: An efficient approach for chart classification

Charts are a visualization tool used in scientific documents to facilitate easy comprehension of complex relationships underlying data and experiments. Researchers use various chart types to convey scientific information, so the problem of data extraction and subsequent chart understanding becomes very challenging. Many studies have been taken up in the literature to address the problem of chart mining, whose motivation is to facilitate the editing of existing charts, carry out extrapolative studies, and provide a deeper understanding of the underlying data. The first step towards chart understanding is chart classification, for which traditional ML and CNN-based deep learning models have been used in the literature. In this paper, we propose Swin-Chart, a Swin transformer-based deep learning approach for chart classification, which generalizes well across multiple datasets with a wide range of chart categories. Swin-Chart comprises a pre-trained Swin Transformer, a finetuning component, and a weight averaging component. The proposed approach is tested on a five-chart image benchmark dataset. We observed that the Swin-Chart model outperformers existing state-of-the-art models on all the datasets. Furthermore, we also provide an ablation study of the Swin-Chart model with all five datasets to understand the importance of various sub-parts such as the back-bone Swin transformer model, the value of several best weights selected for the weight averaging component, and the presence of the weight averaging component itself.The Swin-Chart model also received first position in the chart classification task on the latest dataset in the CHART Infographics competition at ICDAR 2023 - chartinfo.github.io.

Image Denoising Using the Geodesics' Gramian of the Manifold Underlying Patch-Space

With the proliferation of sophisticated cameras in modern society, the demand for accurate and visually pleasing images is increasing. However, the quality of an image captured by a camera may be degraded by noise. Thus, some processing of images is required to filter out the noise without losing vital image features. Even though the current literature offers a variety of denoising methods, the fidelity and efficacy of their denoising are sometimes uncertain. Thus, here we propose a novel and computationally efficient image denoising method that is capable of producing accurate images. To preserve image smoothness, this method inputs patches partitioned from the image rather than pixels. Then, it performs denoising on the manifold underlying the patch-space rather than that in the image domain to better preserve the features across the whole image. We validate the performance of this method against benchmark image processing methods.

An unmanned aerial system benchmark object detection dataset for deep learning in outfall surveys

ABSTRACT Deep-learning-based object detection in UAS imagery is crucial for outfall surveys and basin environmental protection. Comprehensive UAS image datasets serve as a foundation for creating deep-learning-based outfall-detection models and evaluating outfall-detection algorithms. However, existing remote sensing deep learning datasets lack specific outfall data. This study introduces a benchmark UAS image dataset of outfalls, categorized into three main and seven subcategories. Over 10,000 labeled images were collected from UAS images with a 10 cm resolution for the Yangtze River, Yellow River, and other basins from 2019 to 2022, covering nearly all types of outfalls in China. Each sample was matched with digital surface model (DSM) data generated through photographometry or the DSM transfer method. Evaluation with seven widely used deep learning-based object detection algorithms demonstrated the dataset's viability, achieving an average precision ( A P 50 ) of 64.6, surpassing performance on Microsoft Common Objects in Context. Further experiments indicated that DSMs attached to this dataset could benefit from geo-deep-learning-based object detection algorithms for outfalls, achieving an A P 50 exceeding 70. This study presents a UAS benchmark image dataset for outfall surveys, potentially advancing the application of deep learning in environmental protection.

Self-organizing hierarchical incremental learning framework and universal approximation analysis based on stochastic configuration mechanism

Conventional machine learning algorithms face significant limitations when dealing with high-dimensional data. Besides, deep learning models often require substantial computational resources and have a high processing time despite their excellent performance. Hence, this paper proposes an expanded stochastic configuration network and a self-organizing hierarchical incremental learning (SHIL) framework to overcome these challenges. Specifically, this study introduces a novel supervised hierarchical clustering tree based on the minimum redundancy maximum correlation algorithm, which mines internal data structures to construct diverse hierarchies. Subsequently, by exploiting the parent-child node relationships in the tree structure, SHIL defines the maximum number of nodes as the switching condition between levels uses the supervisory mechanism as the parameter selection criterion, and adopts the tolerance error as the termination criterion for the training. Furthermore, the universal approximation property of the SHIL framework is provided. The proposed SHIL framework is validated on several benchmark datasets, image datasets, and industrial robot cases, with the corresponding experimental results demonstrating that SHIL significantly improves computational efficiency and ensures high accuracy.

Neural Network Layer Algebra: A Framework to Measure Capacity and Compression in Deep Learning.

We present a new framework to measure the intrinsic properties of (deep) neural networks. While we focus on convolutional networks, our framework can be extrapolated to any network architecture. In particular, we evaluate two network properties, namely, capacity, which is related to expressivity, and compression, which is related to learnability. Both these properties depend only on the network structure and are independent of the network parameters. To this end, we propose two metrics: the first one, called layer complexity, captures the architectural complexity of any network layer; and, the second one, called layer intrinsic power, encodes how data are compressed along the network. The metrics are based on the concept of layer algebra, which is also introduced in this article. This concept is based on the idea that the global properties depend on the network topology, and the leaf nodes of any neural network can be approximated using local transfer functions, thereby allowing a simple computation of the global metrics. We show that our global complexity metric can be calculated and represented more conveniently than the widely used Vapnik-Chervonenkis (VC) dimension. We also compare the properties of various state-of-the-art architectures using our metrics and use the properties to analyze their accuracy on benchmark image classification datasets.

Image quality and radiation doses in abdominal CT: A multicenter study

PurposeTo benchmark image quality and corresponding radiation doses for acute abdominal CT examination across different laboratories and CT manufacturers. MethodAn anthropomorphic phantom was scanned once with local abdominal CT protocols at 40 CT scanners, from four vendors, in thirty-three sites. Quantitative image quality was evaluated by CNR and SNR in the liver and kidney parenchyma. Qualitative image quality was assessed by visual grading analysis performed by three experienced radiologists using a five-point Likert scale to score thirteen image quality criteria. The CTDIvol was recorded for each scan. Pearson’s correlation coefficient was calculated for the continuous variables, and the intraclass correlation coefficient was used to investigate interrater reliability between the radiologists. ResultsCTDIvol ranged from 3.5 to 12 mGy (median 5.3 mGy, third quartile 6.7 mGy). SNR in liver parenchyma ranged from 4.4 to 14.4 (median 8.5), and CNR ranged from 2.7 to 11.2 (median 6.1). A weak correlation was found between CTDIvol and CNR (r = 0.270, p = 0.092). Variations in CNR across scanners at the same dose level CTDIvol were observed. No significant difference in CTDIvol or CNR was found based on scanner installation year. The oldest scanners had a 15 % higher median CTDIvol and a 12 % lower median CNR. The ICC showed acceptable agreement for all dose groups: low (ICC=0.889), medium (ICC=0.767), high (ICC=0.847), and in low (ICC=0.803) and medium (ICC=0.811) CNR groups. ConclusionThere was large variation in radiation dose and image quality across the different CT scanners. Interestingly, the weak correlation between CTDIvol and CNR indicates that higher doses do not consistently improve CNR, indicating a need for systematic assessment and optimization of image quality and radiation doses for the abdominal CT examination.

ViT-PSO-SVM: Cervical Cancer Predication Based on Integrating Vision Transformer with Particle Swarm Optimization and Support Vector Machine.

Cervical cancer (CCa) is the fourth most prevalent and common cancer affecting women worldwide, with increasing incidence and mortality rates. Hence, early detection of CCa plays a crucial role in improving outcomes. Non-invasive imaging procedures with good diagnostic performance are desirable and have the potential to lessen the degree of intervention associated with the gold standard, biopsy. Recently, artificial intelligence-based diagnostic models such as Vision Transformers (ViT) have shown promising performance in image classification tasks, rivaling or surpassing traditional convolutional neural networks (CNNs). This paper studies the effect of applying a ViT to predict CCa using different image benchmark datasets. A newly developed approach (ViT-PSO-SVM) was presented for boosting the results of the ViT based on integrating the ViT with particle swarm optimization (PSO), and support vector machine (SVM). First, the proposed framework extracts features from the Vision Transformer. Then, PSO is used to reduce the complexity of extracted features and optimize feature representation. Finally, a softmax classification layer is replaced with an SVM classification model to precisely predict CCa. The models are evaluated using two benchmark cervical cell image datasets, namely SipakMed and Herlev, with different classification scenarios: two, three, and five classes. The proposed approach achieved 99.112% accuracy and 99.113% F1-score for SipakMed with two classes and achieved 97.778% accuracy and 97.805% F1-score for Herlev with two classes outperforming other Vision Transformers, CNN models, and pre-trained models. Finally, GradCAM is used as an explainable artificial intelligence (XAI) tool to visualize and understand the regions of a given image that are important for a model's prediction. The obtained experimental results demonstrate the feasibility and efficacy of the developed ViT-PSO-SVM approach and hold the promise of providing a robust, reliable, accurate, and non-invasive diagnostic tool that will lead to improved healthcare outcomes worldwide.

Examining the Application of Deep LSTM Neural Networks in Steganography of Textual Information in Digital Images

Information security has emerged as a critical concern alongside the development of multimedia technology. Among the myriad security challenges, the secure transmission of sensitive information between parties has become a focal point of researchers. Encryption, involving mathematical techniques to ensure data security, is explored in this study. Specifically, the application of deep LSTM neural networks in concealing textual information within digital images is investigated. The approach involves embedding one image within another in a manner that prevents detection of the hidden image within the cover image, while textual content is covertly embedded within the image. The proposed method demonstrates superior performance based on three evaluation metrics—Peak Signal-to-Noise Ratio (PSNR) in decibels, Mean Squared Error (MSE), and accuracy rate in percentage—compared to three other benchmark images (lena.png, peppers.png, mandril.png, and monkey.png), achieving values of 93.665275 dB, 0.6945 MSE, and 97.23% accuracy, respectively.

SCGAN: Semi-Centralized Generative Adversarial Network for image generation in distributed scenes

Distributed Generative Adversarial Networks (GANs) are pivotal for generative tasks in distributed data environments. Conventional architectures, whether centralized or decentralized, inadequately balance global and local data distribution integration. We propose the Semi-Centralized Distributed GAN (SCGAN), a hybrid framework combining the merits of both approaches. SCGAN employs dual generators — one decentralized, one centralized — and a feature extractor that optimizes their outputs by dynamically calculating feature distances, thereby achieving a balanced semi-centralized generator. Extensive experiments on five benchmark image datasets reveal that SCGAN surpasses existing models, delivering superior Fréchet Inception Distance (FID) scores and generating highly realistic images. Moreover, SCGAN’s robustness and versatility are demonstrated through its efficacy in downstream applications, including image augmentation and distributed privacy protection. Notably, SCGAN’s generated images were evaluated using a distributed classifier, a quality metric (IS), and human observation, consistently outperforming existing methods. This study highlights SCGAN’s significant advancements in distributed generative modeling and its nuanced approach to managing complex data distributions, particularly excelling in privacy-preserving scenarios.

Polyp-LVT: Polyp segmentation with lightweight vision transformers

Automatic segmentation of polyps in endoscopic images is crucial for early diagnosis and surgical planning of colorectal cancer. However, polyps closely resemble surrounding mucosal tissue in both texture and indistinct borders and vary in size, appearance, and location which possess great challenge to polyp segmentation. Although some recent attempts have been made to apply Vision Transformer (ViT) to polyp segmentation and achieved promising performance, their application in clinical scenarios is still limited by high computational complexity, large model size, redundant dependencies, and significant training costs. To address these limitations, we propose a novel ViT-based approach named Polyp-LVT, strategically replacing the attention layer in the encoder with a global max pooling layer, which significantly reduces the model’s parameter count and computational cost while keeping the performance undegraded. Furthermore, we introduce a network block, named Inter-block Feature Fusion Module (IFFM), into the decoder, aiming to offer a streamlined yet highly efficient feature extraction. We conduct extensive experiments on three public polyp image benchmarks to evaluate our method. The experimental results show that compared with the baseline models, our Polyp-LVT network achieves a nearly 44% reduction in model parameters while gaining comparable segmentation performance.

Dual-Training-Based Semi-Supervised Learning with Few Labels

The continual expansion in the number of images poses a great challenge for the annotation of the data. Therefore, improving the model performance for image classification with limited labeled data has become an important problem to solve. To address the problem, we propose in this paper a simple and effective dual-training-based semi-supervised learning method for image classification. To enable the model to acquire more valuable information, we propose a dual training approach to enhance model training. Specifically, the model is trained with different augmented data at the same time with soft labels and hard labels, respectively. In addition, we propose a simple and effective weight generation method for generating the weight of samples during training to guide the model training. To further improve the model performance, we employ a projection layer at the end of the network to guide the self-learning of the model by minimizing the distance of features extracted from different layers. Finally, we evaluate the proposed approach on three benchmark image classification datasets. The experimental results demonstrate the effectiveness of our proposed approach.

Neural models for semantic analysis of handwritten document images

Semantic analysis of handwritten document images offers a wide range of practical application scenarios. A sequential combination of handwritten text recognition (HTR) and a task-specific natural language processing system offers an intuitive solution in this domain. However, this HTR-based approach suffers from the problem of error propagation. An HTR-free model, which avoids explicit text recognition and solves the task end-to-end, tackles this problem, but often produces poor results. A possible reason for this is that it does not incorporate largely pre-trained semantic word embeddings, which turn out to be one of the most powerful advantages in the textual domain. In this work, we propose an HTR-based and an HTR-free model and compare them on a variety of segmentation-based handwritten document image benchmarks including semantic word spotting, named entity recognition, and question answering. Furthermore, we propose a cross-modal knowledge distillation approach to integrate semantic knowledge from textually pre-trained word embeddings into HTR-free models. In a series of experiments, we investigate optimization strategies for robust semantic word image representation. We show that the incorporation of semantic knowledge is beneficial for HTR-free approaches in achieving state-of-the-art results on a variety of benchmarks.