Multiple Benchmark Research Articles

Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection

Training neural networks with good generalization requires large computational costs in many deep learning methods due to large-scale datasets and over-parameterized models. Despite the emergence of a number of coreset selection methods to reduce the computational costs, the problem of coreset distribution bias, i.e., the skewed distribution between the coreset and the entire dataset, has not been well studied. In this paper, we find that the closer the feature distribution of the coreset is to that of the entire dataset, the better the generalization performance of the coreset, particularly under extreme pruning. This motivates us to propose a simple yet effective method for coreset selection to alleviate the distribution bias between the coreset and the entire dataset, called feature distribution matching (FDMat). Unlike gradient-based methods, which selects samples with larger gradient values or approximates gradient values of the entire dataset, FDMat aims to select coreset that is closest to feature distribution of the entire dataset. Specifically, FDMat transfers coreset selection as an optimal transport problem from the coreset to the entire dataset in feature embedding spaces. Moreover, our method shows strong robustness due to the removal of samples far from the distribution, especially for the entire dataset containing noisy and class-imbalanced samples. Extensive experiments on multiple benchmarks show that FDMat can improve the performance of coreset selection than existing coreset methods. The code is available at https://github.com/successhaha/FDMat.

ResDiff: Combining CNN and Diffusion Model for Image Super-resolution

Adapting the Diffusion Probabilistic Model (DPM) for direct image super-resolution is wasteful, given that a simple Convolutional Neural Network (CNN) can recover the main low-frequency content. Therefore, we present ResDiff, a novel Diffusion Probabilistic Model based on Residual structure for Single Image Super-Resolution (SISR). ResDiff utilizes a combination of a CNN, which restores primary low-frequency components, and a DPM, which predicts the residual between the ground-truth image and the CNN predicted image. In contrast to the common diffusion-based methods that directly use LR space to guide the noise towards HR space, ResDiff utilizes the CNN’s initial prediction to direct the noise towards the residual space between HR space and CNN-predicted space, which not only accelerates the generation process but also acquires superior sample quality. Additionally, a frequency-domain-based loss function for CNN is introduced to facilitate its restoration, and a frequency-domain guided diffusion is designed for DPM on behalf of predicting high-frequency details. The extensive experiments on multiple benchmark datasets demonstrate that ResDiff outperforms previous diffusion based methods in terms of shorter model convergence time, superior generation quality, and more diverse samples.

Collaborative Consortium of Foundation Models for Open-World Few-Shot Learning

Open-World Few-Shot Learning (OFSL) is a crucial research field dedicated to accurately identifying target samples in scenarios where data is limited and labels are unreliable. This research holds significant practical implications and is highly relevant to real-world applications. Recently, the advancements in foundation models like CLIP and DINO have showcased their robust representation capabilities even in resource-constrained settings with scarce data. This realization has brought about a transformative shift in focus, moving away from “building models from scratch” towards “effectively harnessing the potential of foundation models to extract pertinent prior knowledge suitable for OFSL and utilizing it sensibly”. Motivated by this perspective, we introduce the Collaborative Consortium of Foundation Models (CO3), which leverages CLIP, DINO, GPT-3, and DALL-E to collectively address the OFSL problem. CO3 comprises four key blocks: (1) the Label Correction Block (LC-Block) corrects unreliable labels, (2) the Data Augmentation Block (DA-Block) enhances available data, (3) the Feature Extraction Block (FE-Block) extracts multi-modal features, and (4) the Text-guided Fusion Adapter (TeFu-Adapter) integrates multiple features while mitigating the impact of noisy labels through semantic constraints. Only the adapter's parameters are adjustable, while the others remain frozen. Through collaboration among these foundation models, CO3 effectively unlocks their potential and unifies their capabilities to achieve state-of-the-art performance on multiple benchmark datasets. https://github.com/The-Shuai/CO3.

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and semantic segmentation tasks on multiple benchmark datasets.

Offline Model-Based Optimization via Policy-Guided Gradient Search

Offline optimization is an emerging problem in many experimental engineering domains including protein, drug or aircraft design, where online experimentation to collect evaluation data is too expensive or dangerous. To avoid that, one has to optimize an unknown function given only its offline evaluation at a fixed set of inputs. A naive solution to this problem is to learn a surrogate model of the unknown function and optimize this surrogate instead. However, such a naive optimizer is prone to erroneous overestimation of the surrogate (possibly due to over-fitting on a biased sample of function evaluation) on inputs outside the offline dataset. Prior approaches addressing this challenge have primarily focused on learning robust surrogate models. However, their search strategies are derived from the surrogate model rather than the actual offline data. To fill this important gap, we introduce a new learning-to-search perspective for offline optimization by reformulating it as an offline reinforcement learning problem. Our proposed policy-guided gradient search approach explicitly learns the best policy for a given surrogate model created from the offline data. Our empirical results on multiple benchmarks demonstrate that the learned optimization policy can be combined with existing offline surrogates to significantly improve the optimization performance.

FeatWalk: Enhancing Few-Shot Classification through Local View Leveraging

Few-shot learning is a challenging task due to the limited availability of training samples. Recent few-shot learning studies with meta-learning and simple transfer learning methods have achieved promising performance. However, the feature extractor pre-trained with the upstream dataset may neglect the extraction of certain features which could be crucial for downstream tasks. In this study, inspired by the process of human learning in few-shot tasks, where humans not only observe the whole image (`global view') but also attend to various local image regions (`local view') for comprehensive understanding of detailed features, we propose a simple yet effective few-shot learning method called FeatWalk which can utilize the complementary nature of global and local views, therefore providing an intuitive and effective solution to the problem of insufficient local information extraction from the pre-trained feature extractor. Our method can be easily and flexibly combined with various existing methods, further enhancing few-shot learning performance. Extensive experiments on multiple benchmark datasets consistently demonstrate the effectiveness and versatility of our method.The source code is available at https://github.com/exceefind/FeatWalk.

Multi-Cross Sampling and Frequency-Division Reconstruction for Image Compressed Sensing

Deep Compressed Sensing (DCS) has attracted considerable interest due to its superior quality and speed compared to traditional CS algorithms. However, current approaches employ simplistic convolutional downsampling to acquire measurements, making it difficult to retain high-level features of the original signal for better image reconstruction. Furthermore, these approaches often overlook the presence of both high- and low-frequency information within the network, despite their critical role in achieving high-quality reconstruction. To address these challenges, we propose a novel Multi-Cross Sampling and Frequency Division Network (MCFD-Net) for image CS. The Dynamic Multi-Cross Sampling (DMCS) module, a sampling network of MCFD-Net, incorporates pyramid cross convolution and dual-branch sampling with multi-level pooling. Additionally, it introduces an attention mechanism between perception blocks to enhance adaptive learning effects. In the second deep reconstruction stage, we design a Frequency Division Reconstruction Module (FDRM). This module employs a discrete wavelet transform to extract high- and low-frequency information from images. It then applies multi-scale convolution and self-similarity attention compensation separately to both types of information before merging the output reconstruction results. The MCFD-Net integrates the DMCS and FDRM to construct an end-to-end learning network. Extensive CS experiments conducted on multiple benchmark datasets demonstrate that our MCFD-Net outperforms state-of-the-art approaches, while also exhibiting superior noise robustness.

Traffic pattern-aware elevator dispatching via deep reinforcement learning

This study addresses the elevator dispatching problem using deep reinforcement learning, with a specific emphasis on traffic pattern awareness. Previous studies on reinforcement learning-based elevator dispatching have largely focused on training separate models for single traffic patterns, such as up-peak, down-peak, lunch-peak, and inter-floor. This separate training approach not only introduces practical complexities by requiring an auxiliary model to predict traffic patterns for guiding dispatching decisions but is also computationally burdensome. In contrast, our goal is to develop a unified, traffic pattern-aware dispatching model. We formulate the elevator dispatching problem as a Semi-Markov Decision Process (SMDP) with novel state representation, action space, and reward function designs. To solve the formulated SMDP, we propose a Dueling Double Deep Q-Network (D3QN) architecture associated with the training algorithm. To ensure traffic pattern awareness, we train our model in a unified ‘All in One’ traffic scenario, employing two practical techniques to enhance the training process: (1) temporal grouping with gradient surgery and (2) incorporation of passenger arrival information. Empirical evaluations confirm the superiority of our model over multiple benchmarks, including those relying on separate, pattern-specific models. Remarkably, our unified model demonstrates robust performance across unseen traffic scenarios and performs exceptionally well in single traffic patterns despite being trained solely on the unified ‘All in One’ scenario. The short inference time for decision-making further solidifies the model’s practical viability. Additionally, the incremental benefits contributed by each of our introduced techniques are also investigated. Our code is available at https://github.com/jswan95/RL-based-traffic-pattern-aware-elevator-dispatching

CNN Based Face Emotion Recognition System for Healthcare Application

INTRODUCTION: Because it has various benefits in areas such psychology, human-computer interaction, and marketing, the recognition of facial expressions has gained a lot of attention lately.
 OBJECTIVES: Convolutional neural networks (CNNs) have shown enormous potential for enhancing the accuracy of facial emotion identification systems. In this study, a CNN-based approach for recognizing facial expressions is provided. METHODS: To boost the model's generalizability, transfer learning and data augmentation procedures are applied. The recommended strategy defeated the existing state- of-the-art models when examined on multiple benchmark datasets, including the FER-2013, CK+, and JAFFE databases. 
 RESULTS: The results suggest that the CNN-based approach is fairly excellent at properly recognizing face emotions and has a lot of potential for usage in detecting facial emotions in practical scenarios.
 CONCLUSION: Several diverse forms of information, including oral, textual, and visual, maybe applied to comprehend emotions. In order to increase prediction accuracy and decrease loss, this research recommended a deep CNN model for emotion prediction from facial expression.

Multi-GPU work sharing in a task-based dataflow programming model

Today, multi-GPU computing nodes are the mainstay of most high-performance computing systems. Despite significant progress in programmability, building an application that efficiently utilizes all the GPUs in a computing node is still a significant challenge, especially using the existing shared memory and message-passing paradigms. In this aspect, the task-based dataflow programming model has emerged as an alternative for multi-GPU computing nodes.Most task-based dataflow runtimes have dynamic task mapping, where tasks are mapped to different GPUs based on the current load, but once the mapping has been established, there is no rebalancing of tasks even if an imbalance is detected. In this paper, we examine how automatic dynamic work sharing between GPUs within a compute node can improve the performance of an application through better workload distribution. We demonstrate performance improvement through dynamic work sharing using a Block-Sparse GEneral Matrix Multiplication (BSpGEMM) benchmark. Although we use PaRSEC, a task-based dataflow runtime, as the vehicle for this research, the ideas discussed here are transferable to any task-based dataflow runtime.

Cross-Parallel Attention and Efficient Match Transformer for Aerial Tracking

Visual object tracking is a key technology that is used in unmanned aerial vehicles (UAVs) to achieve autonomous navigation. In recent years, with the rapid development of deep learning, tracking algorithms based on Siamese neural networks have received widespread attention. However, because of complex and diverse tracking scenarios, as well as limited computational resources, most existing tracking algorithms struggle to ensure real-time stable operation while improving tracking performance. Therefore, studying efficient and fast-tracking frameworks, and enhancing the ability of algorithms to respond to complex scenarios has become crucial. Therefore, this paper proposes a cross-parallel attention and efficient match transformer for aerial tracking (SiamEMT). Firstly, we carefully designed the cross-parallel attention mechanism to encode global feature information and to achieve cross-dimensional interaction and feature correlation aggregation via parallel branches, highlighting feature saliency and reducing global redundancy information, as well as improving the tracking algorithm’s ability to distinguish between targets and backgrounds. Meanwhile, we implemented an efficient match transformer to achieve feature matching. This network utilizes parallel, lightweight, multi-head attention mechanisms to pass template information to the search region features, better matching the global similarity between the template and search regions, and improving the algorithm’s ability to perceive target location and feature information. Experiments on multiple drone public benchmark tests verified the accuracy and robustness of the proposed tracker in drone tracking scenarios. In addition, on the embedded artificial intelligence (AI) platform AGX Xavier, our algorithm achieved real-time tracking speed, indicating that our algorithm can be effectively applied to UAV tracking scenarios.

UGNet: Uncertainty aware geometry enhanced networks for stereo matching

Stereo matching is a fundamental research area in the field of computer vision. In recent years, iterative methods based on Gated Recurrent Units (GRUs) have showcased remarkable achievements in this domain. Despite their high accuracy, these methods suffer from notable limitations such as a reliance on a large number of iterations and a tendency to lose high-frequency details. To address these issues, we propose a novel uncertainty-aware framework that combines 3D convolution and GRU-based iterations, aiming to improve efficiency and accuracy. Specifically, we first introduce a probabilistic method to jointly train the disparity map and its corresponding uncertainty map using 3D convolutions. Next, leveraging the uncertainty map as a guide, we develop a novel uncertainty reweighting iterative module to assist in identifying errors in the coarse disparity and cost volume, thereby refining the disparity estimation process and significantly improving the iteration efficiency. Moreover, we introduce a high-resolution refinement module that utilizes Pixel Difference Convolution (PDC) to incorporate additional gradient information. This module can fine-tune the disparity estimation to enhance accuracy. Finally, our network is evaluated on multiple widely-used benchmark datasets. The results demonstrate its proficiency in predicting precise boundaries and effectively reduce iterations. Our model achieves comparable performance to other state-of-the-art methods, ranking 1st on KITTI 2015, and 2nd on KITTI 2012. These results validate its strong performance and generalizability.

Uncertainty-driven active developmental learning

Existing machine learning models can well handle common classes but struggle to detect unfamiliar or unknown classes due to environmental variations. To address this challenge, we propose a new task called active developmental learning (ADL), which empowers models to actively determine what to learn in the open world, thereby progressively enhancing the capability of detecting unfamiliar and unknown classes. Considering the uncertain essence of the task, we design an uncertainty-driven method for ADL (UADL) that measures and utilizes uncertainty to evaluate unfamiliar known classes and unknown classes separately, which consists of two stages: (1) unfamiliar detection of known classes and (2) unknown detection of novel classes. In the first stage, UADL identifies unfamiliar samples of known classes via known-class uncertainty calculated by GMMs on detectors’ heads. In the second stage, UADL identifies samples containing unknown classes via unknown-class uncertainty computed by class-specific GMMs in feature space. In both stages, uncertainty is used to select a minimal number of unlabeled samples for manual labeling, facilitating the model’s active self-development. Experiments on multiple object detection benchmark datasets demonstrate the feasibility and significant performance of UADL and show its effectiveness against the ADL task compared to other state-of-the-art approaches.

Out-of-Domain Generalization From a Single Source: An Uncertainty Quantification Approach.

We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training. We propose Meta-Learning based Adversarial Domain Augmentation to solve this Out-of-Domain generalization problem. The key idea is to leverage adversarial training to create "fictitious" yet "challenging" populations, from which a model can learn to generalize with theoretical guarantees. To facilitate fast and desirable domain augmentation, we cast the model training in a meta-learning scheme and use a Wasserstein Auto-Encoder to relax the widely used worst-case constraint. We further improve our method by integrating uncertainty quantification for efficient domain generalization. Extensive experiments on multiple benchmark datasets indicate its superior performance in tackling single domain generalization.

RAgE: Robust Age Estimation Through Subject Anchoring With Consistency Regularisation.

Modern facial age estimation systems can achieve high accuracy when training and test datasets are identically distributed and captured under similar conditions. However, domain shifts in data, encountered in practice, lead to a sharp drop in accuracy of most existing age estimation algorithms. In this article, we propose a novel method, namely RAgE, to improve the robustness and reduce the uncertainty of age estimates by leveraging unlabelled data through a subject anchoring strategy and a novel consistency regularisation term. First, we propose an similarity-preserving pseudo-labelling algorithm by which the model generates pseudo-labels for a cohort of unlabelled images belonging to the same subject, while taking into account the similarity among age labels. In order to improve the robustness of the system, a consistency regularisation term is then used to simultaneously encourage the model to produce invariant outputs for the images in the cohort with respect to an anchor image. We propose a novel consistency regularisation term the noise-tolerant property of which effectively mitigates the so-called confirmation bias caused by incorrect pseudo-labels. Experiments on multiple benchmark ageing datasets demonstrate substantial improvements over the state-of-the-art methods and robustness to confounding external factors, including subject's head pose, illumination variation and appearance of expression in the face image.

Organ-Agnostic Whole Slide Image Analysis using Self-Supervised Transfer Learning

Traditional methods for pathology image analysis are well-known for their time-consuming and labor-intensive nature, often relying on the expertise of pathologists. In recent years, numerous research studies have been proposed to develop automated systems using machine learning approaches to address these challenges. While these systems have demonstrated promising performance, they often exhibit bias towards specific organs, cells, or tasks, limiting their ability to provide generalized solutions for pathology image analysis. To address this issue, I propose an organ-agnostic pathology image analysis system that leverages a self-supervised transfer learning approach. The proposed system comprises two stages: self-supervised representation learning and transfer learning. In the self-supervised representation learning phase, a machine learning model is trained to consistently extract essential features encapsulating the characteristics of diverse pathological images such as visual patterns of tumors. Subsequently, in the transfer learning phase, these well-pretrained models are utilized to train downstream tasks, such as tumor type classification or cancer area segmentation. The proposed approach outperforms all existing state-of-the-art supervised methods in multiple public pathology image benchmarks.

MolFeSCue: enhancing molecular property prediction in data-limited and imbalanced contexts using few-shot and contrastive learning.

Predicting molecular properties is a pivotal task in various scientific domains, including drug discovery, material science, and computational chemistry. This problem is often hindered by the lack of annotated data and imbalanced class distributions, which pose significant challenges in developing accurate and robust predictive models. This study tackles these issues by employing pretrained molecular models within a few-shot learning framework. A novel dynamic contrastive loss function is utilized to further improve model performance in the situation of class imbalance. The proposed MolFeSCue framework not only facilitates rapid generalization from minimal samples, but also employs a contrastive loss function to extract meaningful molecular representations from imbalanced datasets. Extensive evaluations and comparisons of MolFeSCue and state-of-the-art algorithms have been conducted on multiple benchmark datasets, and the experimental data demonstrate our algorithm's effectiveness in molecular representations and its broad applicability across various pretrained models. Our findings underscore MolFeSCues potential to accelerate advancements in drug discovery. We have made all the source code utilized in this study publicly accessible via GitHub at http://www.healthinformaticslab.org/supp/ or https://github.com/zhangruochi/MolFeSCue. The code (MolFeSCue-v1-00) is also available as the supplementary file of this paper.

Robust Recommender Systems with Rating Flip Noise

Recommender systems have become important tools in the daily life of human beings since they are powerful to address information overload, and discover relevant and useful items for users. The success of recommender systems largely relies on the interaction history between users and items, which is expected to accurately reflect the preferences of users on items. However, the expectation is easily broken in practice, due to the corruptions made in the interaction history, resulting in unreliable and untrusted recommender systems. Previous works either ignore this issue (assume that the interaction history is precise) or are limited to handling additive noise. Motivated by this, in this paper, we study rating flip noise which is widely existed in the interaction history of recommender systems and combat it by modelling the noise generation process. Specifically, the rating flip noise allows a rating to be flipped to any other ratings within the given rating set, which reflects various real-world situations of rating corruption, e.g. , a user may randomly click a rating from the rating set and then submit it. The noise generation process is modelled by the noise transition matrix that denotes the probabilities of a clean rating flip into a noisy rating. A statistically consistent algorithm is afterwards applied with the estimated transition matrix to learn a robust recommender system against rating flip noise. Comprehensive experiments on multiple benchmarks confirm the superiority of our method.

Enhancing the Zebra Optimization Algorithm with Chaotic Sinusoidal Map for Versatile Optimization

In this study, the Chaotic Sinusoidal Map (CSM)-enhanced Zebra Optimization Algorithm (CZOA)is introduced. CZOA combines CSM's integration strengths with ZOA's optimization skills. ZOA already exhibitsgreat optimization capabilities, but the addition of CSM increases its potential even more. This addition greatlystrengthens ZOA's exploration and exploitation skills and increases its flexibility for various optimization tasks.CZOA outperforms both the original ZOA and contemporary optimisation methods on 23 benchmark functions,including high-dimensional (FD), multimodal (MM), and unimodal (UM) challenges. Using the chaos of CSM toinvestigate regional optimal and determine better convergence and exploration-exploitation equilibrium are shownby CZOA, which also shows more profitable solution locations. CZOA demonstrates its resilience and versatilitythrough multiple benchmark activities, underscoring its potential as an adaptable optimisation tool. CZOAbecomes a potent metaheuristic by combining biological inspiration and chaotic dynamics to solve difficultoptimisation problems. Inspired by the natural behaviour of zebras, the Zebra Optimisation Algorithm (ZOA) is arelatively new optimisation technique. It makes use of a herd behaviour mechanism and the ideas of leadership andfollowing, in which members of the population—zebras in this case—cooperate to solve optimisation issues in thebest possible ways

Enhancing medical text detection with vision-language pre-training and efficient segmentation

AbstractDetecting text within medical images presents a formidable challenge in the domain of computer vision due to the intricate nature of textual backgrounds, the dense text concentration, and the possible existence of extreme aspect ratios. This paper introduces an effective and precise text detection system tailored to address these challenges. The system incorporates an optimized segmentation module, a trainable post-processing method, and leverages a vision-language pre-training model (oCLIP). Specifically, our segmentation head integrates three essential components: the Feature Pyramid Network (FPN) module, which combines a residual structure and channel attention mechanism; the Efficient Feature Enhancement Module (EFEM); and the Multi-Scale Feature Fusion with RSEConv (MSFM-RSE), designed specifically for multi-scale feature fusion based on RSEConv. By introducing a residual structure and channel attention mechanism into the FPN module, the convolutional layers are replaced with RSEConv layers that employ a channel attention mechanism, further augmenting the representational capacity of the feature maps. The EFEM, designed as a cascaded U-shaped module, incorporates a spatial attention mechanism to introduce multi-level information, thereby enhancing segmentation performance. Subsequently, the MSFM-RSE adeptly amalgamates features from various depths and scales of the EFEM to generate comprehensive final features tailored for segmentation purposes. Additionally, a post-processing module employs a differentiable binarization strategy, allowing the segmentation network to dynamically determine the binarization threshold. Building on the system’s improvement, we introduce a vision-language pre-training model that undergoes extensive training on various visual language understanding tasks. This pre-trained model acquires detailed visual and semantic representations, further reinforcing both the accuracy and robustness in text detection when integrated with the segmentation module. The performance of our proposed model was evaluated through experiments on medical text image datasets, demonstrating excellent results. Multiple benchmark experiments validate its superior performance in comparison to existing methods. Code is available at: https://github.com/csworkcode/VLDBNet.