Multiple Benchmark Research Articles

Local Patch AutoAugment With Multi-Agent Collaboration

Data augmentation (DA) plays a critical role in improving the generalization of deep learning models. Recent works on automatically searching for DA policies from data have achieved great success. However, existing automated DA methods generally perform the search at the image level, which limits the exploration of diversity in local regions. In this paper, we propose a more fine-grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches. We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image. The agents cooperate with each other to achieve the optimal augmentation effect of the entire image by sharing a team reward. We show the effectiveness of our method on multiple benchmark datasets of image classification, fine-grained image recognition and object detection ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</i> ., CIFAR-10, CIFAR-100, ImageNet, CUB-200-2011, Stanford Cars, FGVC-Aircraft and Pascal VOC 2007). Extensive experiments demonstrate that our method outperforms the state-of-the-art DA methods while requiring fewer computational resources. Automatic Data Augmentation. Multi-Agent Reinforcement Learning.

TF-EPI: an interpretable enhancer-promoter interaction detection method based on Transformer.

The detection of enhancer-promoter interactions (EPIs) is crucial for understanding gene expression regulation, disease mechanisms, and more. In this study, we developed TF-EPI, a deep learning model based on Transformer designed to detect these interactions solely from DNA sequences. The performance of TF-EPI surpassed that of other state-of-the-art methods on multiple benchmark datasets. Importantly, by utilizing the attention mechanism of the Transformer, we identified distinct cell type-specific motifs and sequences in enhancers and promoters, which were validated against databases such as JASPAR and UniBind, highlighting the potential of our method in discovering new biological insights. Moreover, our analysis of the transcription factors (TFs) corresponding to these motifs and short sequence pairs revealed the heterogeneity and commonality of gene regulatory mechanisms and demonstrated the ability to identify TFs relevant to the source information of the cell line. Finally, the introduction of transfer learning can mitigate the challenges posed by cell type-specific gene regulation, yielding enhanced accuracy in cross-cell line EPI detection. Overall, our work unveils important sequence information for the investigation of enhancer-promoter pairs based on the attention mechanism of the Transformer, providing an important milestone in the investigation of cis-regulatory grammar.

A Graph-Based Discriminator Architecture for Multi-Attribute Facial Image Editing

Multi-attribute editing aims to synthesize new facial images with multiple desired attributes while at the same time preserving other contents. Generative Adversarial Networks (GANs) with encoder-decoder-based generators are typically applied to this task, while the co-occurrence nature of attributes is overlooked by generic discriminators when identifying real and synthesized instances. To address the issue, we focus on precisely capturing semantics associated with target attributes in this work, and propose a Graph-based Discriminator architecture for a GAN model, which is referred to as GD-GAN, for explicitly modeling and leveraging the attribute dependencies. Specifically, the co-occurrence ratio between attributes is used to build a correlation matrix, which captures inter-attribute relationships. We design a discriminator with a Graph Convolutional Network (GCN) to integrate knowledge about the attribute dependencies into the adversarial training process. Different from the existing methods that identify the synthesized data conditioned on the attributes individually, we leverage the attribute correlations by performing feature propagation over the graph of attributes, which leads to interdependent representations for real-fake instance identification. Incorporating the relationships of attributes eventually induces the generator to capture precise semantics associated with the attributes. Empirical results on multiple benchmarks demonstrate the superior performance of GD-GAN in high-quality semantic manipulation.

IGMG: Instance-guided multi-granularity for domain generalizable person re-identification

Learning generalized feature embedding is crucial for various computer vision tasks, including domain generalizable person re-identification (ReID). ReID aims to develop deep learning-based feature embeddings that can effectively recognize individuals in both trained (source) domains and unseen target domains. However, many state-of-the-art ReID methods suffer from overfitting as they train and test within the same source domain. To address this issue, we investigate the potential of multi-granularity approaches in mitigating domain shift challenges in person re-identification. Specifically, we propose a novel framework called instance-guided multi-granularity (IGMG), which leverages style-free features through non-parametric Instance Normalization (IN) at multiple granularity levels. While high-level abstract concepts are often not shared across different classes, low- and mid-level features can offer more shareable information to enhance the model’s generalization capabilities. By incorporating this concept, our framework can dynamically eliminate style variations across various levels of abstraction. As a result, it enables the model to capture fine-grained details and high-level semantics, leading to enhanced robustness against changes in data distribution. To validate the effectiveness of our approach, we conduct extensive experiments on multiple benchmark ReID datasets. The results consistently demonstrate that our proposed framework exhibits strong generalization capabilities, performing consistently well on unseen target domains. The code is available at https://github.com/mdamranhossenbhuiyan/IGMG/.

Communication-robust multi-agent learning by adaptable auxiliary multi-agent adversary generation

Communication can promote coordination in cooperative Multi-Agent Reinforcement Learning (MARL). Nowadays, existing works mainly focus on improving the communication efficiency of agents, neglecting that real-world communication is much more challenging as there may exist noise or potential attackers. Thus the robustness of the communication-based policies becomes an emergent and severe issue that needs more exploration. In this paper, we posit that the ego system1) trained with auxiliary adversaries may handle this limitation and propose an adaptable method of Multi-Agent Auxiliary Adversaries Generation for robust Communication, dubbed MA3C, to obtain a robust communication-based policy. In specific, we introduce a novel message-attacking approach that models the learning of the auxiliary attacker as a cooperative problem under a shared goal to minimize the coordination ability of the ego system, with which every information channel may suffer from distinct message attacks. Furthermore, as naive adversarial training may impede the generalization ability of the ego system, we design an attacker population generation approach based on evolutionary learning. Finally, the ego system is paired with an attacker population and then alternatively trained against the continuously evolving attackers to improve its robustness, meaning that both the ego system and the attackers are adaptable. Extensive experiments on multiple benchmarks indicate that our proposed MA3C provides comparable or better robustness and generalization ability than other baselines.

ScInterpreter: a knowledge-regularized generative model for interpretably integrating scRNA-seq data

BackgroundThe rapid emergence of single-cell RNA-seq (scRNA-seq) data presents remarkable opportunities for broad investigations through integration analyses. However, most integration models are black boxes that lack interpretability or are hard to train.ResultsTo address the above issues, we propose scInterpreter, a deep learning-based interpretable model. scInterpreter substantially outperforms other state-of-the-art (SOTA) models in multiple benchmark datasets. In addition, scInterpreter is extensible and can integrate and annotate atlas scRNA-seq data. We evaluated the robustness of scInterpreter in a variety of situations. Through comparison experiments, we found that with a knowledge prior, the training process can be significantly accelerated. Finally, we conducted interpretability analysis for each dimension (pathway) of cell representation in the embedding space.ConclusionsThe results showed that the cell representations obtained by scInterpreter are full of biological significance. Through weight sorting, we found several new genes related to pathways in PBMC dataset. In general, scInterpreter is an effective and interpretable integration tool. It is expected that scInterpreter will bring great convenience to the study of single-cell transcriptomics.

Complex Scenario Image Retrieval via Deep Similarity-aware Hashing

When performing hashing-based image retrieval, it is difficult to learn discriminative hash codes especially for the multi-label, zero-shot and fine-grained settings. This is due to the fact that the similarities vary, even within the same category, under the conditions of complex scenario settings. To address this problem, this study develops a deep similarity-aware hashing method for complex scenario image retrieval (DEPISH). DEPISH more focuses on the samples that are difficult to distinguish from other images (i.e., “difficult samples”), such as images that contain multiple semantics. It dynamically divides attention among samples according to their difficulty levels with a margin weighting strategy. Furthermore, by adding special terms in the model, DEPISH is capable of avoiding the inconsistency between the hash code representation and true similarity among negative samples. In addition, unlike the existing methods that use a pre-defined similarity matrix with fixed values, the DEPISH adopts an adaptive similarity matrix, which accurately captures the various similarities among all samples. The results of our experiment on multiple benchmark datasets containing complex scenarios (i.e., multi-label, zero-shot, and fine-grained datasets) verify the effectiveness of this method.

Discriminative object tracking by domain contrast

Multi-domain tracking method as a novel learning paradigm improves object tracking by sharing domain information with a common backbone whilst learning private information with domain-specific layers. In that context, each individual video sequence as a specific domain serves for a domain-specific layer. In this paper, we observe an intriguing finding that target features from different domains are highly confused with each other, thus having weak discriminative ability, for lack of domain interaction. To this end, we propose a simple yet effective domain interaction training paradigm called domain contrast to boost discriminative object features by effectively using amounts of instances from all the domains in two novel aspects: (1) a light-weight memory-saving training algorithm is proposed to solve the “out-of-the-memory” problem, which paves the way to couple with previous multi-domain trackers, and (2) a composite class-balanced loss is explored to tackle a more practical imbalanced problem, which not only involves the usual class imbalance problem but also accounts for the case of the totally mere negative instances. Experiments on multiple popular tracking benchmark datasets show that our mechanism consistently achieves the tracking performance gain of both base multi-domain tracker and its real-time variant thereof, without any other changes made on the original network.

Self-Improved Learning for Salient Object Detection

Salient Object Detection (SOD) aims at identifying the most visually distinctive objects in a scene. However, learning a mapping directly from a raw image to its corresponding saliency map is still challenging. First, the binary annotations of SOD impede the model from learning the mapping smoothly. Second, the annotator's preference introduces noisy labeling in the SOD datasets. Motivated by these, we propose a novel learning framework which consists of the Self-Improvement Training (SIT) strategy and the Augmentation-based Consistent Learning (ACL) scheme. SIT aims at reducing the learning difficulty, which provides smooth labels and improves the SOD model in a momentum-updating manner. Meanwhile, ACL focuses on improving the robustness of models by regularizing the consistency between raw images and their corresponding augmented images. Extensive experiments on five challenging benchmark datasets demonstrate that the proposed framework can play a plug-and-play role in various existing state-of-the-art SOD methods and improve their performances on multiple benchmarks without any architecture modification.

RepECN: Making ConvNets Better Again for Efficient Image Super-Resolution

RepECN: Making ConvNets Better Again for Efficient Image Super-Resolution

DSLSM: Dual-kernel-induced statistic level set model for image segmentation

Even though the level set method is driving progress in image segmentation, its performance is still affected adversely in the presence of severe intensity inhomogeneity and multi-noise. Therefore, developing an accurate, robust, and fast technique to solve this becomes a pressing need. In this paper, we propose DSLSM, a novel unsupervised statistic level set model based on dual-kernel-induced local similarity measures, which not only can effectively handle severe intensity inhomogeneity and multi-noise, but also stabilizes and accelerates the evolution process. Specifically, we first define a novel dual-kernel-induced data term, which stacks multiple kernel functions and manipulates local similarity measurement via the Jensen–Shannon divergence, aiming at extracting more discriminative higher-dimension features and correlations between adjacent pixels. Then, a novel rectified morphological scheme is proposed to numerically implement our method, which not only precludes the issues of numerical instability and level set function degradation but significantly accelerates the contour evolution. In addition, an inverted pyramid-style scheduling strategy is proposed to explore the complementary utilization of local and non-local contextual features at different evolution phases to avoid tedious heuristics parameter tuning for the size of local regions. Extensive experiments on multiple challenging segmentation benchmarks demonstrate the effectiveness and superiority of our DSLSM over recent state-of-the-art methods.

LibFewShot: A Comprehensive Library for Few-Shot Learning.

Few-shot learning, especially few-shot image classification, has received increasing attention and witnessed significant advances in recent years. Some recent studies implicitly show that many generic techniques or "tricks", such as data augmentation, pre-training, knowledge distillation, and self-supervision, may greatly boost the performance of a few-shot learning method. Moreover, different works may employ different software platforms, backbone architectures and input image sizes, making fair comparisons difficult and practitioners struggle with reproducibility. To address these situations, we propose a comprehensive library for few-shot learning (LibFewShot) by re-implementing eighteen state-of-the-art few-shot learning methods in a unified framework with the same single codebase in PyTorch. Furthermore, based on LibFewShot, we provide comprehensive evaluations on multiple benchmarks with various backbone architectures to evaluate common pitfalls and effects of different training tricks. In addition, with respect to the recent doubts on the necessity of meta- or episodic-training mechanism, our evaluation results confirm that such a mechanism is still necessary especially when combined with pre-training. We hope our work can not only lower the barriers for beginners to enter the area of few-shot learning but also elucidate the effects of nontrivial tricks to facilitate intrinsic research on few-shot learning. The source code is available from https://github.com/RL-VIG/LibFewShot.

Sample-Efficient Cardinality Estimation Using Geometric Deep Learning

In database systems, accurate cardinality estimation is a cornerstone of effective query optimization. In this context, estimators that use machine learning have shown significant promise. Despite their potential, the effectiveness of these learned estimators strongly depends on their ability to learn from small training sets. This paper presents a novel approach for learned cardinality estimation that addresses this issue by enhancing sample efficiency. We propose a neural network architecture informed by geometric deep learning principles that represents queries as join graphs. Furthermore, we introduce an innovative encoding for complex predicates, treating their encoding as a feature selection problem. Additionally, we devise a regularization term that employs equalities of the relational algebra and three-valued logic, augmenting the training process without requiring additional ground truth cardinalities. We rigorously evaluate our model across multiple benchmarks, examining q-errors, runtimes, and the impact of workload distribution shifts. Our results demonstrate that our model significantly improves the end-to-end runtimes of PostgreSQL, even with cardinalities gathered from as little as 100 query executions.

A comparative assessment of the development of GHG emission criteria and benchmark values for buildings in Norway

GHG emission criteria and benchmark values for buildings are being developed internationally to meet climate goals in the Paris agreement. However, there is a distinct lack of harmonisation in approaches making it difficult to compare construction projects or implement and apply benchmarking at a national or international level. Norway is no exception. In recent years, multiple GHG emission criteria and benchmark values have been developed and tested to measure and evaluate the environmental sustainability of Norwegian buildings during their life cycle. These include proposals from Ydalir Masterplan, the research centre for zero emission buildings in smart cities, FutureBuilt ZERO and BREEAM-NOR v6.0. This paper presents and reviews these approaches in relation to recent Norwegian building code requirements for the reporting of GHG emissions. The paper compares the approaches in terms of methodologies, typologies, reference study period, life cycle modules, building parts, limitations and advantages. The results show that there are large differences in approaches which leads to a disparity in benchmarking levels. Further work is required to harmonise and create an accepted branch standard for benchmarking GHG emissions from buildings for the construction industry in Norway. In addition, it would be of benefit to establish a national database for GHG emission accounting so that better benchmark values can be established.

Detect Arbitrary-Shaped Text via Adaptive Thresholding and Localization Quality Estimation

The two-stage scene text detection algorithms based on Mask R-CNN have achieved good performances on multiple challenging benchmarks. However, their effectiveness is degraded due to artificially setting constant thresholds and low localization quality of candidate boxes. In this paper, we present a novel scene text detection method based on Mask R-CNN and the proposed method, named LOAD, proposes adaptive threshold module and localization quality estimation module to address the above two problems. We propose two kinds of adaptive thresholds which are used for the filtering of candidate boxes and the binarization of pixels respectively. We introduce the self-attention mechanism to obtain the global information for generating the adaptive thresholds. Besides, we introduce the localization quality estimation into our model to obtain more accurate candidate boxes for subsequent segmentation. Comparative experiments are conducted on five benchmarks(ICDAR 2015, ICDAR 2017, MSRA-TD500, Total-Text and CTW1500), and the results demonstrate that the proposed method achieves the state-of-the-art performance with an F-measure of 91.0%, 78.7%, 87.4%, 90.6% and 86.0%. We also provide adequate ablation experiments to demonstrate the effectiveness of the proposed components.

Pose-guided Human Feature Aggregation for Occluded Person Re-identification

Since the appearance of most pedestrians is often obscured by various obstacles. Some existing works solve the occlusion problem by aligning the query image of the target pedestrian with the body part of the gallery image, but the body structure of the pedestrian is complicated and not easy to align. Therefore, this paper introduces a Human Feature Aggregation (HFA) approach based on Transformer without alignment, which uses pose information to separate the body parts of target pedestrians from the occlusion. This method utilizes pose information to separate the body parts of the target pedestrian from the obstructions. Initially, the Vision Transformer incorporates Convolutional Neural Network (CNN) advantages to enhance extraction more fine-grained global and local features. Subsequently, the body parts of the target pedestrian are separated from the obstructions using pose information extracted by a pose estimator. Finally, in the human feature aggregation module, local features are matched and fused with pose information to enrich the human features. It steers the model towards focus more on body parts. The experimental findings indicate that the proposed HFA approach surpasses alternative methods on multiple benchmark datasets.

Co-MDA: Federated Multisource Domain Adaptation on Black-Box Models

Federated domain adaptation (FDA) is an effective method for performing over networks from distributed parties, which well improve data privacy and portability in unsupervised multi-source domain adaptation (UMDA) tasks. Despite the impressive gains achieved, two common limitations exist in current FDA works. First, most prior studies require access to the model parameters or gradient details of each source party. However, the raw source data can be reconstructed from the model gradients or parameters, which may leak individual information. Second, these works assume that different parties share an identical network architecture, which is impractical and not desirable for low- or high-resource target users. To address these issues, in this work, we point out a more practical UMDA setting, called Federated Multi-source Domain Adaptation on Black-box Models (B <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FDA), where all data are stored locally and only the input-output interface of the source model is available. To tackle B <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FDA, we propose an effective method, termed Co <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Learning with Multi-Domain Attention (Co-MDA). Experiments on multiple benchmark datasets demonstrate the efficacy of our proposed method. Notably, Co-MDA performs comparably with traditional UMDA methods where the source data or the trained model are fully available.

Enhanced local knowledge with proximity values and syntax-clusters for aspect-level sentiment analysis

Aspect-level sentiment analysis (ALSA) aims to extract the polarity of different aspect terms in a sentence. Previous works leveraging traditional dependency syntax parsing trees (DSPT) to encode contextual syntactic information had obtained state-of-the-art results. However, these works may not be able to learn fine-grained syntactic knowledge efficiently, which makes them difficult to take advantage of local context. Furthermore, these works failed to exploit the dependency relation from DSPT sufficiently. To solve these problems, we propose a novel method to enhance local knowledge by using extensions of Local Context Network based on Proximity Values (LCPV) and Syntax-clusters Attention (SCA), named LCSA. LCPV first gets the induced trees from pre-trained models and generates the syntactic proximity values between context word and aspect to adaptively determine the extent of local context. Our improved SCA further extracts fine-grained knowledge, which not only focuses on the essential clusters for the target aspect term but also guides the model to learn essential words inside each cluster in DSPT. Extensive experimental results on multiple benchmark datasets demonstrate that LCSA is highly robust and achieves state-of-the-art performance for ALSA.

Hybrid Modular Redundancy: Exploring Modular Redundancy Approaches in RISC-V Multi-Core Computing Clusters for Reliable Processing in Space

Space Cyber-Physical Systems (S-CPS) such as spacecraft and satellites strongly rely on the reliability of onboard computers to guarantee the success of their missions. Relying solely on radiation-hardened technologies is extremely expensive, and developing inflexible architectural and microarchitectural modifications to introduce modular redundancy within a system leads to significant area increase and performance degradation. To mitigate the overheads of traditional radiation hardening and modular redundancy approaches, we present a novel Hybrid Modular Redundancy (HMR) approach, a redundancy scheme that features a cluster of RISC-V processors with a flexible on-demand dual-core and triple-core lockstep grouping of computing cores with runtime split-lock capabilities. Further, we propose two recovery approaches, software-based and hardware-based, trading off performance and area overhead. Running at 430MHz, our fault-tolerant cluster achieves up to 1160MOPS on a matrix multiplication benchmark when configured in non-redundant mode and 617 and 414 MOPS in dual and triple mode, respectively. A software-based recovery in triple mode requires 363 clock cycles and occupies 0.612 mm 2 , representing a 1.3% area overhead over a non-redundant 12-core RISC-V cluster. As a high-performance alternative, a new hardware-based method provides rapid fault recovery in just 24 clock cycles and occupies 0.660 mm 2 , namely ∼ 9.4% area overhead over the baseline non-redundant RISC-V cluster. The cluster is also enhanced with split-lock capabilities to enter one of the available redundant modes with minimum performance loss, allowing execution of a mission-critical portion of code when in independent mode, or a performance section when in a reliability mode, with &lt;400 clock cycles overhead for entry and exit. The proposed system is the first to integrate these functionalities on an open-source RISC-V-based compute device, enabling finely tunable reliability vs. performance trade-offs.

Multi-Channel Representation Learning Enhanced Unfolding Multi-Scale Compressed Sensing Network for High Quality Image Reconstruction.

Deep Unfolding Networks (DUNs) serve as a predominant approach for Compressed Sensing (CS) reconstruction algorithms by harnessing optimization. However, a notable constraint within the DUN framework is the restriction to single-channel inputs and outputs at each stage during gradient descent computations. This constraint compels the feature maps of the proximal mapping module to undergo multi-channel to single-channel dimensionality reduction, resulting in limited feature characterization capabilities. Furthermore, most prevalent reconstruction networks rely on single-scale structures, neglecting the extraction of features from different scales, thereby impeding the overall reconstruction network's performance. To address these limitations, this paper introduces a novel CS reconstruction network termed the Multi-channel and Multi-scale Unfolding Network (MMU-Net). MMU-Net embraces a multi-channel approach, featuring the incorporation of Adap-SKConv with an attention mechanism to facilitate the exchange of information between gradient terms and enhance the feature map's characterization capacity. Moreover, a Multi-scale Block is introduced to extract multi-scale features, bolstering the network's ability to characterize and reconstruct the images. Our study extensively evaluates MMU-Net's performance across multiple benchmark datasets, including Urban100, Set11, BSD68, and the UC Merced Land Use Dataset, encompassing both natural and remote sensing images. The results of our study underscore the superior performance of MMU-Net in comparison to existing state-of-the-art CS methods.