Leveraging Afﬁrmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain8Uand Eduardo Blanco7

2025-05-02 0 0 328.01KB 15 页 10玖币

侵权投诉

Leveraging Afﬁrmative Interpretations from Negation

Improves Natural Language Understanding

Md Mosharaf Hossain8,U∗and Eduardo Blanco7

8Department of Computer Science and Engineering, University of North Texas

UAmazon

7Department of Computer Science, University of Arizona

mdmosharafhossain@my.unt.edu hosmdmos@amazon.com eduardoblanco@arizona.edu

Abstract

Negation poses a challenge in many natural lan-

guage understanding tasks. Inspired by the fact

that understanding a negated statement often

requires humans to infer afﬁrmative interpre-

tations, in this paper we show that doing so

beneﬁts models for three natural language un-

derstanding tasks. We present an automated

procedure to collect pairs of sentences with

negation and their afﬁrmative interpretations,

resulting in over 150,000 pairs. Experimental

results show that leveraging these pairs helps

(a) T5 generate afﬁrmative interpretations from

negations in a previous benchmark, and (b) a

RoBERTa-based classiﬁer solve the task of nat-

ural language inference. We also leverage our

pairs to build a plug-and-play neural generator

that given a negated statement generates an af-

ﬁrmative interpretation. Then, we incorporate

the pretrained generator into a RoBERTa-based

classiﬁer for sentiment analysis and show that

doing so improves the results. Crucially, our

proposal does not require any manual effort.

1 Introduction

Natural Language Understanding is a crucial com-

ponent to build intelligent systems that interact with

humans seamlessly. While recent papers some-

times report so-called superhuman performance,

simple adversarial attacks including adding nega-

tion and other input modiﬁcations remain a chal-

lenge despite they are obvious to humans (Naik

et al.,2018;Wallace et al.,2019). Further, many

researchers have found that state-of-the-art systems

struggle with texts containing negation. For ex-

ample, Kassner and Schütze (2020) show that pre-

trained language models such as BERT (Devlin

et al.,2019) do not differentiate between negated

and non-negated cloze questions (e.g., Birds can-

not [MASK] vs. Birds can [MASK]). Other stud-

ies show that transformers perform much worse

in many other natural language understanding

∗Work was done prior to joining Amazon.

English-Norwegian (en-no) parallel sentences:

(en) There is no more than one Truth.

(no) Og det ﬁnnes kun en Sannhet.

Backtranslation: And there is only one truth.

English-Spanish (en-es) parallel sentences:

(en) The term gained traction only after 1999.

(es) El término

se popularizó hasta después del 1999.

Backtranslation: The term was not popular until 1999.

Figure 1: Parallel sentences from bitext corpora

(English-Norwegian and English-Spanish) and back-

translations into English. Either the original English

sentence or the backtranslation contains a negation, and

the other one is an afﬁrmative interpretation. In this

paper, we show that leveraging sentences with nega-

tion and their afﬁrmative interpretations is beneﬁcial for

several natural language understanding tasks including

natural language inference and sentiment analysis.

tasks when there is a negation in the input sen-

tence (Ribeiro et al.,2020;Ettinger,2020;Hossain

et al.,2020b;Hosseini et al.,2021;Hossain et al.,

2022a;Truong et al.,2022).

In this paper, we address this challenge building

upon the following observation: negation often car-

ries afﬁrmative meanings (Horn,1989;Hasson and

Glucksberg,2006). For example, people intuitively

understand that John read part of the book from

John didn’t read the whole book. Our fundamental

idea is to leverage a large collection of sentences

containing negation and their afﬁrmative interpre-

tations. We deﬁne an afﬁrmative interpretation as

a semantically equivalent sentence that does not

contain negation. We explore this idea by automat-

ically collecting pairs of sentences with negation

and their afﬁrmative interpretations from parallel

corpora and backtranslating. Figure 1exempli-

ﬁes the idea with English-Norwegian and English-

Spanish parallel sentences. Note that (a) either the

original English sentence or the backtranslation

have a negation (the one that does not is the afﬁr-

mative interpretation) and (b) the meaning of both

arXiv:2210.14486v1 [cs.CL] 26 Oct 2022

is equivalent.

Armed with the large collection of sentences

containing negation and their afﬁrmative interpre-

tations, we show that leveraging them yields im-

provements in three natural language understand-

ing tasks. First, we address the problem of gen-

erating afﬁrmatively interpretations in the AFIN

benchmark (Hossain et al.,2022b), a collection

of sentences with negation and their manually cu-

rated afﬁrmative interpretations. Second, we ad-

dress natural language inference using three com-

mon benchmarks: RTE (Dagan et al.,2006;Bar-

Haim et al.,2006;Giampiccolo et al.,2007;Ben-

tivogli et al.,2009), SNLI (Bowman et al.,2015),

and MNLI (Williams et al.,2018). Third, we ad-

dress sentiment analysis using SST-2 (Socher et al.,

2013). The main contributions of this paper are:1

A large collection (153,273) of pairs of sen-

tences containing negation and their afﬁrma-

tive interpretations. We present an automated

procedure to get these pairs and an analysis of

the negation types (single tokens, morpholog-

ical, lexicalized, etc.).

Experimental results with the T5 trans-

former (Raffel et al.,2020) showing that

blending our pairs during the ﬁne-tuning pro-

cess is beneﬁcial to generate afﬁrmative inter-

pretations from the negations in AFIN.

Experimental results showing that a

RoBERTa-based classiﬁer (Liu et al.,2019)

to solve the task of natural language inference

beneﬁts from training with new premise-

hypothesis derived from our pairs (two

entailments per pair).

Experimental results showing that a

RoBERTa-based classiﬁer for sentiment

analysis beneﬁts from a novel component

that automatically generates afﬁrmative

interpretations from the input sentence.

The key resource enabling the experimental re-

sults is our large collection of pairs of sentences

containing negation and their afﬁrmative interpre-

tations. As we shall see, the experiments under

(2) and (3) are a somewhat straightforward applica-

tions of these pairs. The afﬁrmative interpretation

generator we use to improve sentiment analysis,

however, has the potential to improve many natural

language understanding tasks.

Code and data available at

https://github.com/

mosharafhossain/large-afin-and-nlu.

2 Related Work

Solving natural language understanding tasks when

the input text contains negation is challenging. Re-

searchers have approached negation processing

mainly by identifying the scope (Vincze et al.,

2008;Morante and Daelemans,2012a) and fo-

cus (Blanco and Moldovan,2011). Scope refers

to the part of the meaning that is negated and fo-

cus refers to the part of the scope that is most

prominently negated (Huddleston and Pullum,

2002). There are many works targeting scope de-

tection (Fancellu et al.,2016,2017;Li and Lu,

2018;Jumelet and Hupkes,2018;Chen,2019;

Zhao and Bethard,2020) and focus detection (Zou

et al.,2014,2015;Shen et al.,2019;Hossain et al.,

2020a). While scope and focus pinpoint what is and

what is not negated, they do not reveal afﬁrmative

interpretations as deﬁned in this paper. Addition-

ally, there is limited empirical evidence showing

that scope or focus is beneﬁcial to solve a natu-

ral language understanding task. Jiménez-Zafra

et al. (2021) show that scope improves sentiment

analysis, but they do not experiment with modern

networks that may not beneﬁt from explicit scope

information.

Outside of scope and focus, Jiang et al. (2021)

work with commonsense implications involving

negations (e.g., “If X doesn’t wear a mask” then “X

is seen as carefree”). Closer to our work, Hosseini

et al. (2021) pretrain BERT with an unlikelihood

loss calculated with automatically obtained negated

statements. Their negated statements do not pre-

serve meaning. The authors show that their method,

BERTNOT, outperforms BERT with LAMA (Kass-

ner and Schütze,2020) and the same natural lan-

guage inference corpora we work with. The work

proposed here outperforms theirs (Section 4.2) and

does not require any manual effort.

We are not the ﬁrst to work with afﬁrmative inter-

pretations from negated statements. For example,

Sarabi et al. (2019) create a small corpus of ver-

bal negations retrieved from Simple Wikipedia and

their afﬁrmative interpretations (total: 5,900). Sim-

ple Wikipedia is a version of Wikipedia that uses

shorter sentences and simpler language. Hossain

et al. (2022b) propose a question-answer driven

approach to create AFIN, a collection of 3,001 sen-

tences with negation and their afﬁrmative inter-

pretations. Both of these previous efforts employ

humans to collect afﬁrmative interpretations and

neither one conducts extrinsic evaluations. Unlike

Source #parl. sents. #pairs %pairs

en-no

WikiMatrix 530,000 10,274 1.94

CCMatrix 8,000,000 73,394 0.92

en-es

UNPC 2,800,000 28,028 1.00

WikiMatrix 3,290,000 41,577 1.26

All 14,620,000 153,273 1.05

Table 1: Number of parallel sentences in the English-

Norwegian and English-Spanish parallel corpora we

work with, and pairs of sentences with negation and

afﬁrmative interpretations we automatically generate

via backtranslation. The yield (%pairs) is low, but as we

shall see these pairs are useful to solve natural language

understanding tasks when negation is present without

hurting results when negation is not present.

them, we automatically collect pairs of sentences

with negation and their afﬁrmative interpretations.

Additionally, extrinsic evaluations show that de-

spite our collection procedure is noisy, leveraging

our pairs is beneﬁcial to solve three natural lan-

guage understanding tasks.

3 Collecting Sentences with Negation

and Their Afﬁrmative Interpretations

This section outlines our approach to create a large

collection of sentences containing negation and

their afﬁrmative interpretations. First, we present

the sources of parallel corpora we work with. Sec-

ond, we describe our multilingual negation cue

detector to identify negation cues in the parallel

sentences. Third, we describe the backtranslation

step and a few checks to improve quality. Lastly,

we present an analysis of the resulting sentences

with negation and their afﬁrmative interpretations.

3.1 Selecting Parallel Corpora

We select parallel sentences in English and either

Norwegian or Spanish for two reasons: (a) large

parallel corpora are available in these language

pairs and (b) negation cue annotations are available

in monolingual corpora for the three languages.

The latter is a requirement to build a multilingual

cue detector (Section 3.2). We extract the paral-

lel sentences from three parallel corpora available

in the OPUS portal (Tiedemann,2012)): WikiMa-

trix (Schwenk et al.,2021a), CCMatrix (Schwenk

et al.,2021b;Fan et al.,2021), and UNPC (Ziemski

et al.,2016). Table 1(Column 3) shows the number

of parallel sentences we collect from each of the

corpora and language pair (total: 14.6 million).

3.2 Identifying Negation Cues in Multiple

Languages

In order to detect negation in the parallel sentences,

we develop a multilingual negation cue detector

that works with English, Norwegian, and Span-

ish texts. To this end, we ﬁne-tune a multilin-

gual BERT (mBERT)

(Devlin et al.,2019) with

negation cue annotations in the three languages

we work with: English (Morante and Daelemans,

2012b), Norwegian (Mæhlum et al.,2021), and

Spanish (Jiménez-Zafra et al.,2018). We ﬁne-tune

jointly for all three languages by combining the

original training splits into a multilingual training

split. We terminate the training process after the

F1 score in the (combined) development split does

not increase for 5 epochs; the ﬁnal model is the

one which yields the highest F1 score during the

training process. Additional details regarding train-

ing procedure and hyperparameters are provided

in Appendix A. Our multilingual detector is not

perfect but obtains competitive results (F1 scores):

English: 91.96 (test split), Norwegian: 93.40 (test

split), and Spanish: 84.41 (dev split, as gold anno-

tations for the test split are not publicly available).

The system detects various negation cue types in-

cluding single tokens (no, never, etc.), afﬁxal, and

lexicalized negations (Section 3.4).

We use our multilingual cue detector to de-

tect negation in the 14.6 million of parallel sen-

tences. In the English-Norwegian parallel sen-

tences (8.5M), negation is present in both sentences

(WikiMatrix: 7.3%, CCMatrix: 14.2%), either

sentence (WikiMatrix: 5.2%, CCMatrix: 5.2%),

or neither sentence (WikiMatrix: 87.5%, CCMa-

trix: 80.6%). Similarly, in English-Spanish parallel

sentences, negation is present in both sentences

(UNPC: 10.7%, WikiMatrix: 5.7%), either sen-

tence (UNPC: 4.6%, WikiMatrix: 4.4%), or nei-

ther sentence (UNPC: 84.7%, WikiMatrix: 89.9%).

Since we are interested in sentences containing

negation and their afﬁrmative interpretations, we

only keep the sentences in which either the source

or target sentence contains negation.

3.3 Generating Afﬁrmative Interpretations

After identifying negation cues in the parallel sen-

tences, we backtranslate into English the sentence

in the target language (either Norwegian or Span-

ish; they may or may not contain a negation). In

2https://github.com/google-research/bert/blob/

master/multilingual.md

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LeveragingAfrmativeInterpretationsfromNegationImprovesNaturalLanguageUnderstandingMdMosharafHossain8,UandEduardoBlanco78DepartmentofComputerScienceandEngineering,UniversityofNorthTexasUAmazon7DepartmentofComputerScience,UniversityofArizonamdmosharafhossain@my.unt.eduhosmdmos@amazon.comeduardoblanc...

展开>> 收起<<

Leveraging Afﬁrmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain8Uand Eduardo Blanco7.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Leveraging Afﬁrmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain8Uand Eduardo Blanco7

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: