Open Access
1- https://doi.org/10.1109/icip51287.2024.10648243
Taxes are All You Need: Integration Of Taxonomical Hierarchy Relationships Into the Contrastive Loss
Translate Article 
In this work, we propose a novel supervised contrastive loss that enables the integration of taxonomic hierarchy information during the representation learning process. A supervised contrastive loss operates by enforcing that images with the same class label (positive samples) project closer to each other than images with differing class labels (negative samples). The advantage of this approach is that it directly penalizes the structure of the representation space itself. This enables greater flexibility with respect to encoding semantic concepts. However, the standard supervised contrastive loss only enforces semantic structure based on the downstream task (i.e. the class label). In reality, the class label is only one level of a hierarchy of different semantic relationships known as a taxonomy. For example, the class label is oftentimes the species of an animal, but between different classes there are higher order relationships such as all animals with wings being "birds". We show that by explicitly accounting for these relationships with a weighting penalty in the contrastive loss we can out-perform the supervised contrastive loss. Additionally, we demonstrate the adaptability of the notion of a taxonomy by integrating our loss into medical and noise-based settings that show performance improvements by as much as 7%.
- Book Chapter
- 10.1007/978-3-031-08473-7_29
- Jan 1, 2022
The relation classification (RC) task classifies a relation present between two target entities in a given context. It is an important task of information extraction and plays a significant role in several NLP applications. Most of the existing studies consider relation classes as a flat list of classes and thus ignore hierarchical relation between classes. This study explores the application of relation hierarchy in improving relation classification performance. In particular, we focus on the following two applications of relation hierarchy: (i) detecting noisy instances; and (ii) modifying the cross-entropy (CE) loss function. We use TACRED, the most widely used RC dataset for this purpose. We build a taxonomical relation hierarchy over the relation classes of TACRED and use it for filtering and relabeling ambiguous or noisy instances of TACRED. For better optimization, we also introduce hierarchical-distance scaled cross-entropy loss (HCE Loss), using the shortest-path distance between ground truth and predicted label for scaling cross-entropy loss. Our extensive empirical analyses indicate that relation hierarchy-inspired filtering, relabeling, and the HCE loss help in improving the relation classification.KeywordsInformation extractionRelation classificationRelation hierarchy
- Research Article
- 37
- 10.1055/s-0038-1634589
- Jan 1, 1995
- Methods of Information in Medicine
A standardized vocabulary and a standardized representation for this vocabulary are necessary prerequisites for the development of a computer-based patient record. A standard conceptual scheme or data structure for this vocabulary must be in place to define clinical events and to share data. SNOMED International is a detailed, fine grained, semantically typed and comprehensive computer processable vocabulary encompassing both human and veterinary medicine. Each term is placed in a standardized data structure that shows the term relationship within its own and other related taxonomic hierarchies. SNOMED International is a standardized vocabulary and data structure suitable for use in the computer-based patient record.
