Benchmark Image Research Articles

MetaStore: Analyzing Deep Learning Meta-Data at Scale

The process of training deep learning models produces a huge amount of meta-data, including but not limited to losses, hidden feature embeddings, and gradients. Model diagnosis tools have been developed to analyze losses and feature embeddings with the aim to improve the performance of these models. However, gradients, despite carrying rich information that is potentially relevant for model interpretation and data debugging, have yet to be fully explored due to their size and complexity. Each single gradient has a size as large as the number of parameters of the neural net - often measured in the tens of millions. This makes it extremely challenging to efficiently collect, store, and analyze large numbers of gradients in these models. In this work, we develop MetaStore to fill this gap. MetaStore leverages our observation that storing certain compact intermediate results produced in the back propagation process, namely, the prefix and suffix gradients, is sufficient for the exact restoration of the original gradient. These prefix and suffix gradients are much more compact than the original gradients, thus allowing us to address the gradient collection and storage challenges. Furthermore, MetaStore features a rich set of analytics operators that allow the users to analyze the gradients for data debugging or model interpretation. Rather than first having to restore the original gradients and then run analytics on top of this decompressed view, MetaStore directly executes these operators on the compact prefix and suffix structures, making gradient-based analytics efficient and scalable. Our experiments on popular deep learning models such as VGG, BERT, and ResNet and benchmark image and text datasets demonstrate that MetaStore outperforms strong baseline methods from 4 to 678x in storage costs and from 2 to 1000x in running time.

Multiview Multilabel Classification With Group-Based Feature and Label Selection

In real-world applications, data are often represented by multiple feature views and associated with multiple labels. In multiview learning, as different views may have noisy and irrelevant features, many works target multiview feature selection, but most of them only perform global feature selection, which means that all samples share the same feature selection weights. In addition, there are many studies in multilabel learning that assume all samples share the same label correlation. However, data may exhibit local patterns, such as feature selection weights and label correlations are locally shared by samples. To address this issue, in this paper, we propose a novel group-based model with local feature and label selection. The proposed model can project samples into different groups by performing group-based feature selection with each view having its own importance for grouping, where each group has its own related labels. The proposed model can then predict the semantics of samples by performing group-based label selection with each group having its own weight for prediction. Besides, the inter-group correlation is also mined and introduced in the above group-based learning process to ensure effective multilabel classification. Empirical studies on multiple image benchmarks validate the effectiveness of the proposed group-based model.

CRAS: Curriculum Regularization and Adaptive Semi-Supervised Learning with Noisy Labels

This paper addresses the performance degradation of deep neural networks caused by learning with noisy labels. Recent research on this topic has exploited the memorization effect: networks fit data with clean labels during the early stages of learning and eventually memorize data with noisy labels. This property allows for the separation of clean and noisy samples from a loss distribution. In recent years, semi-supervised learning, which divides training data into a set of labeled clean samples and a set of unlabeled noisy samples, has achieved impressive results. However, this strategy has two significant problems: (1) the accuracy of dividing the data into clean and noisy samples depends strongly on the network’s performance, and (2) if the divided data are biased towards the unlabeled samples, there are few labeled samples, causing the network to overfit to the labels and leading to a poor generalization performance. To solve these problems, we propose the curriculum regularization and adaptive semi-supervised learning (CRAS) method. Its key ideas are (1) to train the network with robust regularization techniques as a warm-up before dividing the data, and (2) to control the strength of the regularization using loss weights that adaptively respond to data bias, which varies with each split at each training epoch. We evaluated the performance of CRAS on benchmark image classification datasets, CIFAR-10 and CIFAR-100, and real-world datasets, mini-WebVision and Clothing1M. The findings demonstrate that CRAS excels in handling noisy labels, resulting in a superior generalization and robustness to a range of noise rates, compared with the existing method.

Source Camera Identification Techniques: A Survey.

The successful investigation and prosecution of significant crimes, including child pornography, insurance fraud, movie piracy, traffic monitoring, and scientific fraud, hinge largely on the availability of solid evidence to establish the case beyond any reasonable doubt. When dealing with digital images/videos as evidence in such investigations, there is a critical need to conclusively prove the source camera/device of the questioned image. Extensive research has been conducted in the past decade to address this requirement, resulting in various methods categorized into brand, model, or individual image source camera identification techniques. This paper presents a survey of all those existing methods found in the literature. It thoroughly examines the efficacy of these existing techniques for identifying the source camera of images, utilizing both intrinsic hardware artifacts such as sensor pattern noise and lens optical distortion, and software artifacts like color filter array and auto white balancing. The investigation aims to discern the strengths and weaknesses of these techniques. The paper provides publicly available benchmark image datasets and assessment criteria used to measure the performance of those different methods, facilitating a comprehensive comparison of existing approaches. In conclusion, the paper outlines directions for future research in the field of source camera identification.

Empowering breast cancer diagnosis and radiology practice: advances in artificial intelligence for contrast-enhanced mammography.

Artificial intelligence (AI) applications in breast imaging span a wide range of tasks including decision support, risk assessment, patient management, quality assessment, treatment response assessment and image enhancement. However, their integration into the clinical workflow has been slow due to the lack of a consensus on data quality, benchmarked robust implementation, and consensus-based guidelines to ensure standardization and generalization. Contrast-enhanced mammography (CEM) has improved sensitivity and specificity compared to current standards of breast cancer diagnostic imaging i.e., mammography (MG) and/or conventional ultrasound (US), with comparable accuracy to MRI (current diagnostic imaging benchmark), but at a much lower cost and higher throughput. This makes CEM an excellent tool for widespread breast lesion characterization for all women, including underserved and minority women. Underlining the critical need for early detection and accurate diagnosis of breast cancer, this review examines the limitations of conventional approaches and reveals how AI can help overcome them. The Methodical approaches, such as image processing, feature extraction, quantitative analysis, lesion classification, lesion segmentation, integration with clinical data, early detection, and screening support have been carefully analysed in recent studies addressing breast cancer detection and diagnosis. Recent guidelines described by Checklist for Artificial Intelligence in Medical Imaging (CLAIM) to establish a robust framework for rigorous evaluation and surveying has inspired the current review criteria.

Learning the Frequency Domain Aliasing for Real-World Super-Resolution

Most real-world super-resolution methods require synthetic image pairs for training. However, the frequency domain gap between synthetic images and real-world images leads to artifacts and blurred reconstructions. This work points out that the main reason for the frequency domain gap is that aliasing exists in real-world images, but the degradation model used to generate synthetic images ignores the impact of aliasing on images. Therefore, a method is proposed in this work to assess aliasing in images undergoing unknown degradation by measuring the distance to their alias-free counterparts. Leveraging this assessment, a domain-translation framework is introduced to learn degradation from high-resolution to low-resolution images. The proposed framework employs a frequency-domain branch and loss function to generate synthetic images with aliasing features. Experiments validate that the proposed domain-translation framework enhances the visual quality and quantitative results compared to existing super-resolution models across diverse real-world image benchmarks. In summary, this work offers a practical solution to the real-world super-resolution problem by minimizing the frequency domain gap between synthetic and real-world images.

An efficient hybrid differential evolution-golden jackal optimization algorithm for multilevel thresholding image segmentation.

Image segmentation is a crucial process in the field of image processing. Multilevel threshold segmentation is an effective image segmentation method, where an image is segmented into different regions based on multilevel thresholds for information analysis. However, the complexity of multilevel thresholding increases dramatically as the number of thresholds increases. To address this challenge, this article proposes a novel hybrid algorithm, termed differential evolution-golden jackal optimizer (DEGJO), for multilevel thresholding image segmentation using the minimum cross-entropy (MCE) as a fitness function. The DE algorithm is combined with the GJO algorithm for iterative updating of position, which enhances the search capacity of the GJO algorithm. The performance of the DEGJO algorithm is assessed on the CEC2021 benchmark function and compared with state-of-the-art optimization algorithms. Additionally, the efficacy of the proposed algorithm is evaluated by performing multilevel segmentation experiments on benchmark images. The experimental results demonstrate that the DEGJO algorithm achieves superior performance in terms of fitness values compared to other metaheuristic algorithms. Moreover, it also yields good results in quantitative performance metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and feature similarity index (FSIM) measurements.

Hierarchical Locality-Aware Deep Dictionary Learning for Classification

Deep dictionary learning (DDL) shows good performance in visual classification tasks. However, almost all existing DDL methods ignore the locality relationships between the input data representations and the learned dictionary atoms, and learn sub-optimal representations in the feature coding stage, which are less conducive to classification. To this end, we propose a hierarchical locality-aware deep dictionary learning (HILADLE) framework for classification, which can learn locality-constrained dictionaries at different abstract levels through hierarchical dictionary learning. The locality constraints play an important role in learning informative dictionary atoms while preserving the data structure in the original input feature space. Moreover, instead of using an identity activation function like existing DDL methods, we further boost the generalization performance of our HILADLE method with a ReLU activation function to deal with the overfitting issue caused by over-parameterization, inspired by its effectiveness in deep neural networks. Finally, the concatenation of all feature representations learned at different layers is used as input to the final classifier. We demonstrate, through an extensive set of experiments on several benchmark face recognition, image classification, and age estimation datasets, that our method is able to surpass several dictionary learning, deep dictionary learning and deep learning methods.

A Gating Model for Bias Calibration in Generalized Zero-shot Learning.

Generalized zero-shot learning (GZSL) aims at training a model that can generalize to unseen class data by only using auxiliary information. One of the main challenges in GZSL is a biased model prediction toward seen classes caused by overfitting on only available seen class data during training. To overcome this issue, we propose a two-stream autoencoder-based gating model for GZSL. Our gating model predicts whether the query data is from seen classes or unseen classes, and utilizes separate seen and unseen experts to predict the class independently from each other. This framework avoids comparing the biased prediction scores for seen classes with the prediction scores for unseen classes. In particular, we measure the distance between visual and attribute representations in the latent space and the cross-reconstruction space of the autoencoder. These distances are utilized as complementary features to characterize unseen classes at different levels of data abstraction. Also, the two-stream autoencoder works as a unified framework for the gating model and the unseen expert, which makes the proposed method computationally efficient. We validate our proposed method in four benchmark image recognition datasets. In comparison with other state-of-the-art methods, we achieve the best harmonic mean accuracy in SUN and AWA2, and the second best in CUB and AWA1. Furthermore, our base model requires at least 20% less number of model parameters than state-of-the-art methods relying on generative models.

Development of an Approach for Image Forgery Detection Using Machine Learning Algorithms

Digital picture fraud detection is an increasing societal necessity due to the importance of verified images. The detection of picture copying, splicing, retouching, and re-sampling forgeries is included. In the absence of digital signatures or watermarks, passive picture authentication may serve as an alternative to active authentication. Passive techniques, every so often recognized as blind techniques, could take place without preceding knowledge of the picture or its reference. Identifying counterfeiting picture or tampering was a research field for long a period of time, triggered via the Internet, online platforms, social messaging platforms, and extensive digital image usage. The rate of failure could be a key factor for examining the alteration of picture or forgery, among other existing methods. The research applies almost six common algorithms related to machine learning in order to extract features from Lightweight, Spatial Exploitation, and Residual deep learning models on benchmark datasets MICC-F220, Columbia, and CoMoFoD. The models of incorporated deep learning could consist of AlexNet, GoogleNet, VGG16, VGG19, SqueezeNet, MobileNetV2, ShuffleNet, ResNet-18, ResNet-50, and ResNet-101 for spatial exploitation. Fine-tuning is applied to the top three deep learning models, optimizing hyperparameters centered on indicators of performance for every single benchmark dataset. Tweaked SqueezeNet, MobileNetV2, and ShuffleNet deep learning models with SGDM Optimizer and SVM classifier yielded the best results for MICC-F220 dataset. Fine-tuned VGG19, MobileNetV2, and ResNet-50 deep learning models with SGDM Optimizer and SVM v classifier yielded the best results for Columbia dataset. In CoMoFoD dataset, fine-tuned AlexNet, MobileNetV2, and ShuffleNet deep learning models with SGDM Optimizer and SVM classifier yielded the best results. The proposed approach, utilizing machine learning algorithms and deep learning features, enhanced forgery detection and reduced false positives. Results were validated on benchmark image forgery datasets and compared to current methods.

Stable tensor neural networks for efficient deep learning.

Learning from complex, multidimensional data has become central to computational mathematics, and among the most successful high-dimensional function approximators are deep neural networks (DNNs). Training DNNs is posed as an optimization problem to learn network weights or parameters that well-approximate a mapping from input to target data. Multiway data or tensors arise naturally in myriad ways in deep learning, in particular as input data and as high-dimensional weights and features extracted by the network, with the latter often being a bottleneck in terms of speed and memory. In this work, we leverage tensor representations and processing to efficiently parameterize DNNs when learning from high-dimensional data. We propose tensor neural networks (t-NNs), a natural extension of traditional fully-connected networks, that can be trained efficiently in a reduced, yet more powerful parameter space. Our t-NNs are built upon matrix-mimetic tensor-tensor products, which retain algebraic properties of matrix multiplication while capturing high-dimensional correlations. Mimeticity enables t-NNs to inherit desirable properties of modern DNN architectures. We exemplify this by extending recent work on stable neural networks, which interpret DNNs as discretizations of differential equations, to our multidimensional framework. We provide empirical evidence of the parametric advantages of t-NNs on dimensionality reduction using autoencoders and classification using fully-connected and stable variants on benchmark imaging datasets MNIST and CIFAR-10.

Class-Incremental Learning: A Survey.

Deep models, e.g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world. However, novel classes emerge from time to time in our ever-changing world, requiring a learning system to acquire new knowledge continually. Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally and build a universal classifier among all seen classes. Correspondingly, when directly training the model with new class instances, a fatal problem occurs - the model tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. There have been numerous efforts to tackle catastrophic forgetting in the machine learning community. In this paper, we survey comprehensively recent advances in class-incremental learning and summarize these methods from several aspects. We also provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms empirically. Furthermore, we notice that the current comparison protocol ignores the influence of memory budget in model storage, which may result in unfair comparison and biased results. Hence, we advocate fair comparison by aligning the memory budget in evaluation, as well as several memory-agnostic performance measures. The source code is available at https://github.com/zhoudw-zdw/CIL_Survey/.

Adaptive Annotation Correlation Based Multi-Annotation Learning for Calibrated Medical Image Segmentation.

Medical image segmentation is a fundamental task in many clinical applications, yet current automated segmentation methods rely heavily on manual annotations, which are inherently subjective and prone to annotation bias. Recently, modeling annotator preference has garnered great interest, and several methods have been proposed in the past two years. However, the existing methods completely ignore the potential correlation between annotations, such as complementary and discriminative information. In this work, the Adaptive annotation CorrelaTion based multI-annOtation LearNing (ACTION) method is proposed for calibrated medical image segmentation. ACTION employs consensus feature learning and dynamic adaptive weighting to leverage complementary information across annotations and emphasize discriminative information within each annotation based on their correlations, respectively. Meanwhile, memory accumulation-replay is proposed to accumulate the prior knowledge and integrate it into the model to enable the model to accommodate the multi-annotation setting. Two medical image benchmarks with different modalities are utilized to evaluate the performance of ACTION, and extensive experimental results demonstrate that it achieves superior performance compared to several state-of-the-art methods.

Fractal Few-Shot Learning.

Forming deep feature embeddings is an effective method for few-shot learning (FSL). However, in the case of insufficient samples, overcoming the task complexity while improving the accuracy is still a major challenge. To address this problem, this article considers the consistency between similar data from the fractal perspective, introduces a priori knowledge, and proposes a fractal embedding model by combining FSL with fractal dimension theory for the first time. We improve the original fractal dimension algorithm used to describe image texture roughness to suit a neural network. Moreover, in accordance with the improved algorithm, prior knowledge of the quantized image is integrated into the features to reduce the impact of the data distribution on the model. Experimental results obtained on multiple image benchmark datasets show that the performance of the proposed model exceeds or matches that of previous state-of-the-art models. In addition, the proposed model achieves the best performance in cross-domain scenarios, further illustrating its robustness.

IMPLEMENTASI METODE SERVQUAL DALAM MENGUKUR TINGKAT KEPUASAN PASIEN TERHADAP KUALITAS PELAYANAN RAWAT JALAN Studi Kasus: Puskesmas Maubesi

The service quality of a Community Health Center does have a great impact on the patients' satisfaction. The Maubesi Community Health Center is one of the health services centers in Maubesi Village which serves a number of six surrounding villages. Given that many patients are making use of this health center, the service quality of the medical staff is of course quite essential. Patients' satisfaction is the primary factor which becomes the image benchmark of a community health center. Up to now, patients' satisfaction is measured in the conventional way, namely by writing on papers. The use òf paper certainly makes it hectic when the time for data compilation and analysis processing is due. To overcome this problem, the researcher developed a website-based information system (SISKESAN) to measure patients' satisfaction level at Maubesi Community Health Center. SISKESAN was developed based on website, to make it easy whenever and wherever the data is to be accessed. The system employs Service Quality Method (Servqual) to measure patients' satisfaction level using hope and fact values. To set up this system the researcher made use of Waterfall Method known as measurable gradual stages. By implementing SISKESAN at Maubesi Community Health Center, the patients' satisfaction level becomes easier in measurement. Patients can easily access the website and fill in the questionnaires prepared in the system. The result can be straightly seen by the staff in charge and it will become the benchmark for future policy making.

Unsupervised Image Hashing Using a Deep Convolutional Encoder-Decoder Model for Fast Image Retrieval

Image hashing methods transform high-dimensional image features into low-dimensional binary codes while preserving semantic similarity. Among image hashing techniques, supervised image hashing approaches outperform unsupervised and semisupervised methods. However, labelling image data requires extra time and expert effort. In this study, we proposed a deep learning-based unsupervised image hashing method for unlabeled image data. The proposed hashing method is built in an end-to-end fashion. It consists of an encoder-decoder model. As a novel idea, we used a supervised pre-trained network as an encoder model, which provides fast convergence in the training phase and efficient image features. Hash codes are extracted by optimizing those intermediate features. Experiments performed on two benchmark image datasets demonstrate the competitive results compared to unsupervised image hashing methods.

An Underwater Image Restoration Deep Learning Network Combining Attention Mechanism and Brightness Adjustment

This study proposes Combining Attention and Brightness Adjustment Network (CABA-Net), a deep learning network for underwater image restoration, to address the issues of underwater image color-cast, low brightness, and low contrast. The proposed approach achieves a multi-branch ambient light estimation by extracting the features of different levels of underwater images to achieve accurate estimates of the ambient light. Additionally, an encoder-decoder transmission map estimation module is designed to combine spatial attention structures that can extract the different layers of underwater images’ spatial features to achieve accurate transmission map estimates. Then, the transmission map and precisely predicted ambient light were included in the underwater image formation model to achieve a preliminary restoration of underwater images. HSV brightness adjustment was conducted by combining the channel and spatial attention to the initial underwater image to complete the final underwater image restoration. Experimental results on the Underwater Image Enhancement Benchmark (UIEB) and Real-world Underwater Image Enhancement (RUIE) datasets show excellent performance of the proposed method in subjective comparisons and objective assessments. Furthermore, several ablation studies are conducted to understand the effect of each network component and prove the effectiveness of the suggested approach.

Learning from partially labeled data for multi-organ and tumor segmentation.

Medical image benchmarks for the segmentation of organs and tumors suffer from the partially labeling issue due to its intensive cost of labor and expertise. Current mainstream approaches follow the practice of one network solving one task. With this pipeline, not only the performance is limited by the typically small dataset of a single task, but also the computation cost linearly increases with the number of tasks. To address this, we propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple partially labeled datasets. Specifically, TransDoDNet has a hybrid backbone that is composed of the convolutional neural network and Transformer. A dynamic head enables the network to accomplish multiple segmentation tasks flexibly. Unlike existing approaches that fix kernels after training, the kernels in the dynamic head are generated adaptively by the Transformer, which employs the self-attention mechanism to model long-range organ-wise dependencies and decodes the organ embedding that can represent each organ. We create a large-scale partially labeled Multi-Organ and Tumor Segmentation benchmark, termed MOTS, and demonstrate the superior performance of our TransDoDNet over other competitors on seven organ and tumor segmentation tasks. This study also provides a general 3D medical image segmentation model, which has been pre-trained on the large-scale MOTS benchmark and has demonstrated advanced performance over current predominant self-supervised learning methods. Code and data are available at https://github.com/jianpengz/DoDNet.

MiAMix: Enhancing Image Classification through a Multi-Stage Augmented Mixed Sample Data Augmentation Method

Despite substantial progress in the field of deep learning, overfitting persists as a critical challenge, and data augmentation has emerged as a particularly promising approach due to its capacity to enhance model generalization in various computer vision tasks. While various strategies have been proposed, Mixed Sample Data Augmentation (MSDA) has shown great potential for enhancing model performance and generalization. We introduce a novel mixup method called MiAMix, which stands for Multi-stage Augmented Mixup. MiAMix integrates image augmentation into the mixup framework, utilizes multiple diversified mixing methods concurrently, and improves the mixing method by randomly selecting mixing mask augmentation methods. Recent methods utilize saliency information and the MiAMix is designed for computational efficiency as well, reducing additional overhead and offering easy integration into existing training pipelines. We comprehensively evaluate MiAMix using four image benchmarks and pitting it against current state-of-the-art mixed sample data augmentation techniques to demonstrate that MiAMix improves performance without heavy computational overhead.

OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning

Federated learning (FL) has emerged as a promising approach to collaboratively train machine learning models across multiple edge devices while preserving privacy. The success of FL hinges on the efficiency of participating models and their ability to handle the unique challenges of distributed learning. While several variants of Vision Transformer (ViT) have shown great potential as alternatives to modern convolutional neural networks (CNNs) for centralized training, the unprecedented size and higher computational demands hinder their deployment on resource-constrained edge devices, challenging their widespread application in FL. Since client devices in FL typically have limited computing resources and communication bandwidth, models intended for such devices must strike a balance between model size, computational efficiency, and the ability to adapt to the diverse and non-IID data distributions encountered in FL. To address these challenges, we propose OnDev-LCT: Lightweight Convolutional Transformers for On-Device vision tasks with limited training data and resources. Our models incorporate image-specific inductive biases through the LCT tokenizer by leveraging efficient depthwise separable convolutions in residual linear bottleneck blocks to extract local features, while the multi-head self-attention (MHSA) mechanism in the LCT encoder implicitly facilitates capturing global representations of images. Extensive experiments on benchmark image datasets indicate that our models outperform existing lightweight vision models while having fewer parameters and lower computational demands, making them suitable for FL scenarios with data heterogeneity and communication bottlenecks.