Long Tail Research Articles

Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

The performance of semantic segmentation in remote sensing, based on deep learning models, depends on the training data. A commonly encountered issue is the imbalanced long-tailed distribution of data, where the head classes contain the majority of samples while the tail classes have fewer samples. When training with long-tailed data, the head classes dominate the training process, resulting in poorer performance in the tail classes. To address this issue, various strategies have been proposed, such as resampling, reweighting, and transfer learning. However, common resampling methods suffer from overfitting to the tail classes while underfitting the head classes, and reweighting methods are limited in the extreme imbalanced case. Additionally, transfer learning tends to transfer patterns learned from the head classes to the tail classes without rigorously validating its generalizability. These methods often lack additional information to assist in the recognition of tail class objects, thus limiting performance improvements and constraining generalization ability. To tackle the abovementioned issues, a graph neural network based on the graph kernel principle is proposed for the first time. By leveraging the graph kernel, structural information for tail class objects is obtained, serving as additional contextual information beyond basic visual features. This method partially compensates for the imbalance between tail and head class object information without compromising the recognition accuracy of head classes objects. The experimental results demonstrate that this study effectively addresses the poor recognition performance of small and rare targets, partially alleviates the issue of spectral confusion, and enhances the model’s generalization ability.

ORDERED SEARCH, INTRAPLATFORM COMPETITION AND AN INVISIBLE MARKET BOUNDARY—EVIDENCE FROM CHINESE E-COMMERCE PLATFORMS

Using data from three leading e-commerce platforms in China, we study the impact of the dispersion of sellers’ ranking positions on the price dispersion of homogeneous commodities. The major findings are as follows: The dispersion of the ranking positions of sellers weakens market competition and intensifies price dispersion. The dispersion of ranking positions has a heterogeneous effect on sellers’ pricing strategies with different ranking positions. In addition, under the combined action of search costs and long-tail effects in e-commerce markets, there is an invisible market boundary in online markets.

Learning-to-rank debias with popularity-weighted negative sampling and popularity regularization

Most learning-to-rank recommendation models assume users prefer interacted items more than non-interacted ones. All non-interacted items have the same chance to be selected as negative samples. However, the observed interaction data usually shows long-tailed distributions, making traditional recommendation models trained on such data tend to rank popular items higher than users’ true preference and degrading the performance of recommender systems. Some existing debias methods focus on eliminating such popularity bias, while simply removing all the popularity bias may diminish recommendation accuracy because popularity also contains some useful information implying item attractiveness, making it pivotal to handling popularity bias in proper ways. In this paper, popularity is recognized to have both positive and negative effects on recommendation. The positive effect of popularity should be amplified while the negative effect should be mitigated. Therefore, this paper proposes two novel schemes to distinguish the positive and negative impact of popularity in learning-to-rank models. First, we propose a popular-weighted sampler to sample non-interacted popular items from users’ collaborative items as negative samples with higher probabilities. We believe that if a popular item is collaboratively similar to a user but not interacted with by him, it is more likely to be a true negative sample for the user. We sample such items with higher probabilities to mitigate the harmful negative popularity bias. Second, popularity could be a significant indicator of items’ intrinsic quality. Hence, we design a popularity regularization scheme to capture item attractiveness, prompting cold-start users to prefer popular items. Finally, we conduct extensive experiments to evaluate the performance of the proposed schemes. Experimental results demonstrate that our approach can improve existing learning-to-rank models’ performance and outperform state-of-the-art debias methods.

Particle-in-cell Simulations of the Magnetorotational Instability in Stratified Shearing Boxes

Abstract The magnetorotational instability (MRI) plays a crucial role in regulating the accretion efficiency in astrophysical accretion disks. In low-luminosity disks around black holes, such as Sgr A* and M87, Coulomb collisions are infrequent, making the MRI physics effectively collisionless. The collisionless MRI gives rise to kinetic plasma effects that can potentially affect its dynamic and thermodynamic properties. We present 2D and 3D particle-in-cell (PIC) plasma simulations of the collisionless MRI in stratified disks using shearing boxes with net vertical field. We use pair plasmas, with initial β = 100 and concentrate on sub-relativistic plasma temperatures (kBT ≲ mc2). Our 2D and 3D runs show disk expansion, particle and magnetic field outflows, and a dynamo-like process. They also produce magnetic pressure dominated disks with (Maxwell stress dominated) viscosity parameter α ∼ 0.5 − 1. By the end of the simulations, the dynamo-like magnetic field tends to dominate the magnetic energy and the viscosity in the disks. Our 2D and 3D runs produce fairly similar results, and are also consistent with previous 3D MHD simulations. Our simulations also show nonthermal particle acceleration, approximately characterized by power-law tails with temperature dependent spectral indices −p. For temperatures kBT ∼ 0.05 − 0.3 mc2, we find p ≈ 2.2 − 1.9. The maximum accelerated particle energy depends on the scale separation between MHD and Larmor-scale plasma phenomena in a way consistent with previous PIC results of magnetic reconnection-driven acceleration. Our study constitutes a first step towards modeling from first principles potentially observable stratified MRI effects in low-luminosity accretion disks around black holes.

PLoPS: Localization-aware person search with prototypical normalization

Person search involves localizing and re-identifying persons of interest captured by multiple, non-overlapping cameras. Recent approaches to person search are typically built on object detection frameworks to learn joint person representations for detection and re-identification. To this end, the features extracted from pedestrian proposals are projected onto a unit hypersphere using L2 normalization, and positive proposals that sufficiently overlap with the ground truth are equally incorporated for training by exploiting an external lookup table (LUT). We have found that (1) using the L2 normalization technique, without considering feature distributions, can degenerate the discriminative power of person representations, (2) positive proposals often depict distracting details, such as background clutter and person overlaps, and (3) person features in the LUT are not often updated during training. To address these limitations, we propose a novel framework for person search, dubbed PLoPS, using a prototypical normalization layer, ProtoNorm, that calibrates features while considering the long-tail distribution across person IDs. PLoPS also entails a localization-aware learning scheme that prioritizes better-aligned proposals w.r.t the ground truth. We further introduce a LUT calibration technique to continuously adjust the person features in the LUT. Experimental results and analysis on standard benchmarks demonstrate the effectiveness of PLoPS.

CFD-DEM-based evaluation of main-channel sediment transport processes subject to supplement from a steep tributary

The accumulation of sediments within mountainous gullies serves as a significant source for the main channel. Understanding the transport characteristics is crucial for comprehending reach-scale sediment supply features and channel morphodynamics. This study addresses the previously unexplored challenges in accurately quantifying variations in sediment transport rates, particularly during critical transitions such as the mobilization of static sediment pulses into continuous motion. To accomplish this objective, a combined Computational Fluid Dynamics and Discrete Element Method (CFD-DEM) computational framework is employed to directly simulate the detailed trajectories of all particles. After validating the CFD-DEM model, we examine the effects of three major influential factors (particle size, channel slope and channel width) on sediment transport characteristics. The results indicate that the sediment transport rate exhibits rapid rise to peak followed by gradual attenuation process, described by a long-tail effect. The cross-sectional averaged sediment transport rate positively correlates with the width of sediment transport belt. The downstream propagation of sediment is enhanced by steeper channel slope and narrower channel width. In addition, we find that cross-sectional averaged sediment transport rate controlled by these factors exhibits inherent self-similarity. The temporal variation of dimensionless sediment transport rate for a certain section can be modeled by a modified two-branch Weibull function encompasses three model parameters dependent on influential factors and longitudinal location. The parameter variation of the functional model, combined with linear variation of the three main influencing factors, provides support for explaining sediment transport processes in channels of dimensions different from those studied. These findings enhance the understanding of the formation and evolution mechanism of sediment transport induced by upstream loose deposit destruction.

Learning the long-tail distribution in latent space for Weighted Link Prediction via conditional Invertible Neural Networks

Link prediction is an important problem in dynamic systems, where the goal is to predict future unweighted or weighted link topologies using historical context. Compared with unweighted links, weighted links can preferably reveal the nature and strength of the interactions among entities. However, weighted links also bring greater challenges because they require subtle structural adjustments and numerical variations to be captured. Existing methods are primarily tailored for unweighted links and most generally suffer from low-quality performance when applied to Weighted Link Prediction (WLP) task. In this study, we propose a novel generative framework that adopts conditional Invertible Neural Networks (INNs) to achieve WLP. The proposed framework leverages the benefits of conditional INNs to exactly optimize the log-likelihood in the latent space conditioned on the historical context, which can be sensitive to minor replacements in real-world systems and derive accurate WLPs. Furthermore, to deal with the long-tail statistical phenomenon of edge weights observed in real life, a tail-adaptive distribution is learned in latent space to capture the tail properties and enhance the model’s ability. To verify the effectiveness of the proposed method, we conduct extensive experiments on four datasets from different systems. The experimental results demonstrate that our model achieves impressive results compared to state-of-the-art competitors.

Turbulent Magnetic Reconnection as an Acceleration Mechanism in Earth’s Magnetotail

Using electric and magnetic fields measured by the Magnetospheric Multiscale (MMS) mission, we construct a test-particle simulation of a turbulent magnetic reconnection region to investigate observed ion acceleration. We identify three types of energized ions: (1) ion jets, (2) Speiser-like energized ions—both of which carry significant energy but do not produce a strong energetic (>80 keV) tail in the ion distribution—and (3) a separate but sizable population of ions that are accelerated to significantly higher energies (>80 keV) by the turbulent fields. The majority of ions that undergo energization by the turbulent fields cross the magnetic null plane multiple times. By preferentially energizing these particles, the turbulence creates a separate population of ions that mostly exits in the dawn direction of the magnetotail and forms a high-energy power-law tail in the ion flux-energy distribution. We also find that the highest acceleration energies are limited by the size of the turbulent region (with respect to ion gyroradii). Since turbulence is widespread in astrophysical plasmas and has no a priori limit on scale size, the MMS observations suggest turbulence may have a significant role in particle acceleration.

On the Value of Head Labels in Multi-Label Text Classification

A formidable challenge in the multi-label text classification (MLTC) context is that the labels often exhibit a long-tailed distribution, which typically prevents deep MLTC models from obtaining satisfactory performance. To alleviate this problem, most existing solutions attempt to improve tail performance by means of sampling or introducing extra knowledge. Data-rich labels, though more trustworthy, have not received the attention they deserve. In this work, we propose a multiple-stage training framework to exploit both model- and feature-level knowledge from the head labels, to improve both the representation and generalization ability of MLTC models. Moreover, we theoretically prove the superiority of our framework design over other alternatives. Comprehensive experiments on widely used MLTC datasets clearly demonstrate that the proposed framework achieves highly superior results to state-of-the-art methods, highlighting the value of head labels in MLTC.

SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation

Image-level weakly supervised semantic segmentation has received increasing attention due to its low annotation cost. Existing methods mainly rely on Class Activation Mapping (CAM) to obtain pseudo-labels for training semantic segmentation models. In this work, we are the first to demonstrate that long-tailed distribution in training data can cause the CAM calculated through classifier weights over-activated for head classes and under-activated for tail classes due to the shared features among head- and tail- classes. This degrades pseudo-label quality and further influences final semantic segmentation performance. To address this issue, we propose a Shared Feature Calibration (SFC) method for CAM generation. Specifically, we leverage the class prototypes which carry positive shared features and propose a Multi-Scaled Distribution-Weighted (MSDW) consistency loss for narrowing the gap between the CAMs generated through classifier weights and class prototypes during training. The MSDW loss counterbalances over-activation and under-activation by calibrating the shared features in head-/tail-class classifier weights. Experimental results show that our SFC significantly improves CAM boundaries and achieves new state-of-the-art performances. The project is available at https://github.com/Barrett-python/SFC.

Inverse Weight-Balancing for Deep Long-Tailed Learning

The performance of deep learning models often degrades rapidly when faced with imbalanced data characterized by a long-tailed distribution. Researchers have found that the fully connected layer trained by cross-entropy loss has large weight-norms for classes with many samples, but not for classes with few samples. How to address the data imbalance problem with both the encoder and the classifier seems an under-researched problem. In this paper, we propose an inverse weight-balancing (IWB) approach to guide model training and alleviate the data imbalance problem in two stages. In the first stage, an encoder and classifier (the fully connected layer) are trained using conventional cross-entropy loss. In the second stage, with a fixed encoder, the classifier is finetuned through an adaptive distribution for IWB in the decision space. Unlike existing inverse image frequency that implements a multiplicative margin adjustment transformation in the classification layer, our approach can be interpreted as an adaptive distribution alignment strategy using not only the class-wise number distribution but also the sample-wise difficulty distribution in both encoder and classifier. Experiments show that our method can greatly improve performance on imbalanced datasets such as CIFAR100-LT with different imbalance factors, ImageNet-LT, and iNaturelists2018.

CLIP-Guided Federated Learning on Heterogeneity and Long-Tailed Data

Federated learning (FL) provides a decentralized machine learning paradigm where a server collaborates with a group of clients to learn a global model without accessing the clients' data. User heterogeneity is a significant challenge for FL, which together with the class-distribution imbalance further enhances the difficulty of FL. Great progress has been made in large vision-language models, such as Contrastive Language-Image Pre-training (CLIP), which paves a new way for image classification and object recognition. Inspired by the success of CLIP on few-shot and zero-shot learning, we use CLIP to optimize the federated learning between server and client models under its vision-language supervision. It is promising to mitigate the user heterogeneity and class-distribution balance due to the powerful cross-modality representation and rich open-vocabulary prior knowledge. In this paper, we propose the CLIP-guided FL (CLIP2FL) method on heterogeneous and long-tailed data. In CLIP2FL, the knowledge of the off-the-shelf CLIP model is transferred to the client-server models, and a bridge is built between the client and server. Specifically, for client-side learning, knowledge distillation is conducted between client models and CLIP to improve the ability of client-side feature representation. For server-side learning, in order to mitigate the heterogeneity and class-distribution imbalance, we generate federated features to retrain the server model. A prototype contrastive learning with the supervision of the text encoder of CLIP is introduced to generate federated features depending on the client-side gradients, and they are used to retrain a balanced server classifier. Extensive experimental results on several benchmarks demonstrate that CLIP2FL achieves impressive performance and effectively deals with data heterogeneity and long-tail distribution. The code is available at https://github.com/shijiangming1/CLIP2FL.

Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning

Compositional Zero-Shot Learning (CZSL) aims to transfer knowledge from seen state-object pairs to novel unseen pairs. In this process, visual bias caused by the diverse interrelationship of state-object combinations blurs their visual features, hindering the learning of distinguishable class prototypes. Prevailing methods concentrate on disentangling states and objects directly from visual features, disregarding potential enhancements that could arise from a data viewpoint. Experimentally, we unveil the results caused by the above problem closely approximate the long-tailed distribution. As a solution, we transform CZSL into a proximate class imbalance problem. We mathematically deduce the role of class prior within the long-tailed distribution in CZSL. Building upon this insight, we incorporate visual bias caused by compositions into the classifier's training and inference by estimating it as a proximate class prior. This enhancement encourages the classifier to acquire more discernible class prototypes for each composition, thereby achieving more balanced predictions. Experimental results demonstrate that our approach elevates the model's performance to the state-of-the-art level, without introducing additional parameters.

Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization

Domain Generalization (DG) aims to improve the generalization ability of models trained on a specific group of source domains, enabling them to perform well on new, unseen target domains. Recent studies have shown that methods that converge to smooth optima can enhance the generalization performance of supervised learning tasks such as classification. In this study, we examine the impact of smoothness-enhancing formulations on domain adversarial training, which combines task loss and adversarial loss objectives. Our approach leverages the fact that converging to a smooth minimum with respect to task loss can stabilize the task loss and lead to better performance on unseen domains. Furthermore, we recognize that the distribution of objects in the real world often follows a long-tailed class distribution, resulting in a mismatch between machine learning models and our expectations of their performance on all classes of datasets with long-tailed class distributions. To address this issue, we consider the domain generalization problem from the perspective of the long-tail distribution and propose using the maximum square loss to balance different classes which can improve model generalizability. Our method's effectiveness is demonstrated through comparisons with state-of-the-art methods on various domain generalization datasets. Code: https://github.com/bamboosir920/SAMALTDG.

Intentional Evolutionary Learning for Untrimmed Videos with Long Tail Distribution

Human intention understanding in untrimmed videos aims to watch a natural video and predict what the person’s intention is. Currently, exploration of predicting human intentions in untrimmed videos is far from enough. On the one hand, untrimmed videos with mixed actions and backgrounds have a significant long-tail distribution with concept drift characteristics. On the other hand, most methods can only perceive instantaneous intentions, but cannot determine the evolution of intentions. To solve the above challenges, we propose a loss based on Instance Confidence and Class Accuracy (ICCA), which aims to alleviate the prediction bias caused by the long-tail distribution with concept drift characteristics in video streams. In addition, we propose an intention-oriented evolutionary learning method to determine the intention evolution pattern (from what action to what action) and the time of evolution (when the action evolves). We conducted extensive experiments on two untrimmed video datasets (THUMOS14 and ActivityNET v1.3), and our method has achieved excellent results compared to SOTA methods. The code and supplementary materials are available at https://github.com/Jennifer123www/UntrimmedVideo.

EAT: Towards Long-Tailed Out-of-Distribution Detection

Despite recent advancements in out-of-distribution (OOD) detection, most current studies assume a class-balanced in-distribution training dataset, which is rarely the case in real-world scenarios. This paper addresses the challenging task of long-tailed OOD detection, where the in-distribution data follows a long-tailed class distribution. The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes, as the ability of a classifier to detect OOD instances is not strongly correlated with its accuracy on the in-distribution classes. To overcome this issue, we propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes. This approach allows us to build a detector with clear decision boundaries by training on OOD data using virtual labels. (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data. This technique encourages the model to pay more attention to the discriminative features of the tail classes. We provide a clue for separating in-distribution and OOD data by analyzing gradient noise. Through extensive experiments, we demonstrate that our method outperforms the current state-of-the-art on various benchmark datasets. Moreover, our method can be used as an add-on for existing long-tail learning approaches, significantly enhancing their OOD detection performance. Code is available at: https://github.com/Stomach-ache/Long-Tailed-OOD-Detection.

Long-Tailed Learning as Multi-Objective Optimization

Real-world data is extremely imbalanced and presents a long-tailed distribution, resulting in models biased towards classes with sufficient samples and performing poorly on rare classes. Recent methods propose to rebalance classes but they undertake the seesaw dilemma (what is increasing performance on tail classes may decrease that of head classes, and vice versa). In this paper, we argue that the seesaw dilemma is derived from the gradient imbalance of different classes, in which gradients of inappropriate classes are set to important for updating, thus prone to overcompensation or undercompensation on tail classes. To achieve ideal compensation, we formulate long-tailed recognition as a multi-objective optimization problem, which fairly respects the contributions of head and tail classes simultaneously. For efficiency, we propose a Gradient-Balancing Grouping (GBG) strategy to gather the classes with similar gradient directions, thus approximately making every update under a Pareto descent direction. Our GBG method drives classes with similar gradient directions to form a more representative gradient and provides ideal compensation to the tail classes. Moreover, we conduct extensive experiments on commonly used benchmarks in long-tailed learning and demonstrate the superiority of our method over existing SOTA methods. Our code is released at https://github.com/WickyLee1998/GBG_v1.

Dual-Prior Augmented Decoding Network for Long Tail Distribution in HOI Detection

Human object interaction detection aims at localizing human-object pairs and recognizing their interactions. Trapped by the long-tailed distribution of the data, existing HOI detection methods often have difficulty recognizing the tail categories. Many approaches try to improve the recognition of HOI tasks by utilizing external knowledge (e.g. pre-trained visual-language models). However, these approaches mainly utilize external knowledge at the HOI combination level and achieve limited improvement in the tail categories. In this paper, we propose a dual-prior augmented decoding network by decomposing the HOI task into two sub-tasks: human-object pair detection and interaction recognition. For each subtask, we leverage external knowledge to enhance the model's ability at a finer granularity. Specifically, we acquire the prior candidates from an external classifier and embed them to assist the subsequent decoding process. Thus, the long-tail problem is mitigated from a coarse-to-fine level with the corresponding external knowledge. Our approach outperforms existing state-of-the-art models in various settings and significantly boosts the performance on the tail HOI categories. The source code is available at https://github.com/PRIS-CV/DP-ADN.

DME: Unveiling the Bias for Better Generalized Monocular Depth Estimation

This paper aims to design monocular depth estimation models with better generalization abilities. To this end, we have conducted quantitative analysis and discovered two important insights. First, the Simulation Correlation phenomenon, commonly seen in long-tailed classification problems, also exists in monocular depth estimation, indicating that the imbalanced depth distribution in training data may be the cause of limited generalization ability. Second, the imbalanced and long-tail distribution of depth values extends beyond the dataset scale, and also manifests within each individual image, further exacerbating the challenge of monocular depth estimation. Motivated by the above findings, we propose the Distance-aware Multi-Expert (DME) depth estimation model. Unlike prior methods that handle different depth range indiscriminately, DME adopts a divide-and-conquer philosophy where each expert is responsible for depth estimation of regions within a specific depth range. As such, the depth distribution seen by each expert is more uniform and can be more easily predicted. A pixel-level routing module is further designed and learned to stitch the prediction of all experts into the final depth map. Experiments show that DME achieves state-of-the-art performance on both NYU-Depth v2 and KITTI, and also delivers favorable zero-shot generalization capability on unseen datasets.

Multi-Modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation

3D point cloud semantic segmentation has a wide range of applications. Recently, weakly supervised point cloud segmentation methods have been proposed, aiming to alleviate the expensive and laborious manual annotation process by leveraging scene-level labels. However, these methods have not effectively exploited the rich geometric information (such as shape and scale) and appearance information (such as color and texture) present in RGB-D scans. Furthermore, current approaches fail to fully leverage the point affinity that can be inferred from the feature extraction network, which is crucial for learning from weak scene-level labels. Additionally, previous work overlooks the detrimental effects of the long-tailed distribution of point cloud data in weakly supervised 3D semantic segmentation. To this end, this paper proposes a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module. The point affinity proposed in this paper is characterized by features from multiple modalities (e.g., point cloud and RGB), and is further refined by normalizing the classifier weights to alleviate the detrimental effects of long-tailed distribution without the need of the prior of category distribution. Extensive experiments on the ScanNet and S3DIS benchmarks verify the effectiveness of our proposed method, which outperforms the state-of-the-art by ~4% to ~ 6% mIoU. Codes are released at https://github.com/Sunny599/AAAI24-3DWSSG-MMA.