Abstract

Sarcasm and sentiment embody intrinsic uncertainty of human cognition, making joint detection of multi-modal sarcasm and sentiment a challenging task. In view of the advantages of quantum probability (QP) in modeling such uncertainty, this paper explores the potential of QP as a mathematical framework and proposes a QP driven multi-task (QPM) learning framework. The QPM framework involves a complex-valued multi-modal representation encoder, a quantum-like fusion subnetwork and a quantum measurement mechanism. Each multi-modal (e.g., textual, visual) utterance is first encoded as a quantum superposition of a set of basis terms using a complex-valued representation. Then, the quantum-like fusion subnetwork leverages quantum state composition and quantum interference to model the contextual interaction between adjacent utterances and the correlations across modalities respectively. Finally, quantum incompatible measurements are performed on the multi-modal representation of each utterance to yield the probabilistic outcomes of sarcasm and sentiment recognition. The experimental results show that our model achieves a state-of-the-art performance.

Highlights

  • Motivated by recent success in using quantum probability (QP) as a formal framework for modeling the intrinsic uncertainty in human cognition, we take the first step towards using QP to solve the joint multi-modal sarcasm and sentiment analyof each utterance to yield the probabilistic outcomes of sarcasm and sentiment recognition

  • ∗Yazhou Zhang and Yaochen Liu contribute and share the co-first authorship. †Corresponding author complex probability amplitude, and models an utterance as a quantum superposition of basis words or pixels; (2) Quantum interference embodies a an interference term for modeling two decision paths interfering with each other in reaching a final decision (e.g. 2 Quantum Probability Preliminaries sarcasm judgment); (3) Quantum contextuality reflects the intra-modality contextual interaction as quantum composition; (4) Quantum incompatible measurement describes the correlations across multiple tasks

  • QP driven multi-task (QPM), and we propose to perform a sequence of quantum incompatible measurements on |fp, for obtaining the sarcastic and sentimental probabilistic features msar and msen

Read more

Summary

Introduction

Motivated by recent success in using quantum probability (QP) as a formal framework for modeling the intrinsic uncertainty in human cognition, we take the first step towards using QP to solve the joint multi-modal sarcasm and sentiment analyof each utterance to yield the probabilistic outcomes of sarcasm and sentiment recognition. 2018a; Li et al, 2019) and sentiment classification (Zhang et al, 2020; Gkoumas et al, 2021), with verified effectiveness and advantages Different from these existing approaches, at the heart of our work are quantum inspired modeling of multimodal fusion in conversational context and exploring the inter-task correlations via quantum incompatible measurement. An interference term for modeling two decision paths (e.g., textual and visual modalities) interfering with each other in reaching a final decision (e.g. 2 Quantum Probability Preliminaries sarcasm judgment); (3) Quantum contextuality reflects the intra-modality contextual interaction as quantum composition; (4) Quantum incompatible measurement describes the correlations across multiple tasks.

Objectives
Methods
Results
Discussion
Conclusion

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.