Abstract
The lexical substitution task aims at generating a list of suitable replacements for a target word in context, ideally keeping the meaning of the modified text unchanged. While its usage has increased in recent years, the paucity of annotated data prevents the finetuning of neural models on the task, hindering the full fruition of recently introduced powerful architectures such as language models. Furthermore, lexical substitution is usually evaluated in a framework that is strictly bound to a limited vocabulary, making it impossible to credit appropriate, but out-of-vocabulary, substitutes. To assess these issues, we proposed GeneSis (Generating Substitutes in contexts), the first generative approach to lexical substitution. Thanks to a seq2seq model, we generate substitutes for a word according to the context it appears in, attaining state-of-the-art results on different benchmarks. Moreover, our approach allows silver data to be produced for further improving the performances of lexical substitution systems. Along with an extensive analysis of GeneSis results, we also present a human evaluation of the generated substitutes in order to assess their quality. We release the fine-tuned models, the generated datasets, and the code to reproduce the experiments at https://github.com/SapienzaNLP/genesis.
Highlights
Despite having been employed in numerous downstream tasks, the lexical substitution task still presents unresolved issues that need to be mances of lexical substitution systems
As source dataset C we exploit SemCor (Miller et al, 1993), a manually annotated corpus where instances are sense-tagged according to the WordNet sense inventory4. While it is typically used as a training corpus for English Word Sense Disambiguation (WSD), as we show, its manually-curated sense distribution is beneficial for lexical substitution
Evaluation Metrics We evaluate the performance of our model using the metrics originally proposed for the task
Summary
The task of substitutes prediction requires finding ing time, we provide the model with a sequence of gold substitutes s = s1, . We can train M replacements for a target word in a context that by minimizing the cross-entropy loss between the ideally do not modify the overall meaning of that gold and the generated sequences. Given a target word wt occurring in a context sentence x = w1, . Wn, a substitution system has to assemble a ranked list time, for each input sequence mwt,x we produce several substitute sequences s1, . Sb obtained with beam-search decoding (Figure 1(a)). S of possible replacements for wt according to its context x.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
