Mutual Information Alleviates Hallucinations in Abstractive Summarization Liam van der Poel Ryan Cotterell Clara Meister

2025-05-02 0 0 516.26KB 10 页 10玖币

侵权投诉

Mutual Information Alleviates Hallucinations in Abstractive

Summarization

Liam van der Poel Ryan Cotterell Clara Meister

lvander@ethz.ch {ryan.cotterell,clara.meister}@inf.ethz.ch

Abstract

Despite signiﬁcant progress in the quality of

language generated from abstractive summa-

rization models, these models still exhibit the

tendency to hallucinate, i.e., output content not

supported by the source document. A num-

ber of works have tried to ﬁx—or at least un-

cover the source of—the problem with limited

success. In this paper, we identify a simple

criterion under which models are signiﬁcantly

more likely to assign more probability to hallu-

cinated content during generation: high model

uncertainty. This ﬁnding offers a potential ex-

planation for hallucinations: models default to

favoring text with high marginal probability,

i.e., high-frequency occurrences in the train-

ing set, when uncertain about a continuation.

It also motivates possible routes for real-time

intervention during decoding to prevent such

hallucinations. We propose a decoding strat-

egy that switches to optimizing for pointwise

mutual information of the source and target

token—rather than purely the probability of

the target token—when the model exhibits un-

certainty. Experiments on the XSUM dataset

show that our method decreases the probability

of hallucinated tokens while maintaining the

ROUGE and BERTS scores of top-performing

decoding strategies.

https://github.com/VanderpoelLiam/

CPMI

1 Introduction

Abstractive summarization, the task of condensing

long documents into short summaries, has a num-

ber of applications, such as providing overviews of

news articles or highlighting main points in tech-

nical documents. Abstractive summarization is

usually performed using probabilistic text gener-

ators (Goyal and Durrett,2020;Mao et al.,2020;

Kryscinski et al.,2020), which have shown a strong

ability to produce ﬂuent, human-like text (Baevski

and Auli,2019;Radford et al.,2019;Brown et al.,

2020). However, these models have been observed

hallucinate

facts, i.e., add information to the out-

put that was not present in the original text. This

behavior is problematic, as presenting users with

unsubstantiated content can lead to undesirable ef-

fects, such as the spread of misinformation (Bender

et al.,2021;Abid et al.,2021;Liang et al.,2021).

Some works have attributed this phenomenon to

the speciﬁc training corpora for these models, in

which ground-truth summaries often contain out-

side information that may not have been directly

deducible from the original text (Maynez et al.,

2020;Zhou et al.,2021). Others have pointed to

model architectures or training strategies (Voita

et al.,2021;Wang and Sennrich,2020;Kang and

Hashimoto,2020). While these works have given

us an improved understanding of the cause of hal-

lucinations, there still does not exist an efﬁcient

and robust set of techniques for identifying and

preventing them during the generation process.

This work aims to ﬁrst provide a simple criterion

indicating when a model is more likely to assign

higher probability to content not necessarily de-

rived from the source document. Speciﬁcally, we

link the start of a hallucination during generation to

high model uncertainty about the next token, which

we quantify by conditional entropy. We hypothe-

size that hallucinations may be due to a tendency of

models to default to placing probability mass on to-

kens that appeared frequently in the training corpus,

a behavior by language models previously observed

in several natural language processing (NLP) tasks

(Kobayashi et al.,2020;Wei et al.,2021). As a con-

sequence, generations with hallucinations would

still be viable candidates, as standard decoding

strategies for summarization optimize purely for

the probability of the generation. We propose an al-

ternative decoding strategy to combat this behavior:

When a model exhibits high uncertainty, we change

our decoding objective to pointwise mutual infor-

mation between the source document and target

token (PMI; Li et al.,2016;Takayama and Arase,

2019), encouraging the model to prioritize tokens

relevant to the source document. While changing

arXiv:2210.13210v2 [cs.CL] 29 Oct 2022

completely to the PMI objective causes a drop of

3.13%

in ROUGE-L scores, this conditional and

temporary change leads to only a

0.977%

drop in

ROUGE-L while increasing factuality according to

the FACTScore metric.

In experiments, we ﬁrst observe a strong correla-

tion between conditional entropy and the start of a

hallucination on an annotated subset of the XSUM

dataset (Maynez et al.,2020). We next score the tar-

gets in the annotated subset under both the standard

log-probability objective and CPMI, and observe

that the revised log-probability of hallucinated to-

kens under the CPMI objective is indeed lower .

Finally, we ﬁnd that our proposed decoding strat-

egy maintains ROUGE and BERTS scores.

2 Preliminaries

In this work, we consider probabilistic models

for abstractive summarization. Explicitly, we con-

sider models with distribution

p(y|x)

, where

is the source document that we wish to summa-

rize and

y=hy0, . . . , yTi

is a string, represented

as a sequence of tokens from the model’s vocab-

ulary

. The set of valid sequences

is then de-

ﬁned as all sequences

such that

def

=BOS

and

def

=EOS

, the beginning- and end-of-sequence

tokens, respectively, and

yt∈ V

for

0< t < T

Note that standard models are locally normalized,

i.e., they provide a probability distribution over

Vdef

=V ∪ {EOS}

at time step

given the source

document and prior context

p(· | y<t,x)

. The

probability of an entire string

can then be com-

puted as

p(y|x) = QT

t=1 p(yt|y<t,x)

, where

for shorthand we deﬁne y<t

def

=hy0, . . . , yt−1i.

Generation from

is performed token-by-token

due to the autoregressive natures of most language

generators. We typically seek to generate a string

that maximizes some score function

y?= argmax

y∈Y

score(y|x)(1)

In the case of probabilistic models, this function is

often simply

score(y|x) = log p(y|x)

, i.e., we

want to generate a high probability string

. Note

that searching over the entire space

is usually

infeasible (or at least impractical) due to the non-

Markovian nature of most neural models. Thus we

often use an approximate search algorithm such

as beam search, as given in Alg 1, that optimizes

for our score function somewhat greedily. This

procedure meshes well with the use of

log p

Algorithm 1

Standard beam search.

1◦

used to rep-

resent string concatenation.

Input: x: source document

k: maximum beam size

nmax: maximum hypothesis length

score(·|·): scoring function

1: B0← {h0,BOSi} → beam set

2: for t∈ {1, . . . , nmax}do

3: B← ∅

4: for hs, yi ∈ Bt−1do

5: for y∈ V do

6: s←score(y◦y|x)

7: B.add(hs, y◦yi)

8: Bt←B.top(k)

9: return Bnmax .max()

the score function since it can be decomposed as

the sum of individual token log-probabilities, i.e.,

we can instead consider a token-wise score func-

tion

score(y|y<t,x) = log p(y|y<t,x)

using

the fact that

score(y|x) = PT

t=1 score(y|y<t,x)

We only consider decoding strategies for score

functions that can be decomposed in this manner.

Evaluation.

Abstractive summarization systems

are usually evaluated using automatic metrics, such

as ROUGE (Lin,2004). While ROUGE generally

correlates poorly with human judgments (Maynez

et al.,2020;Fabbri et al.,2021) and is only weakly

correlated with factuality,

it is quick to com-

pute, making it useful for quickly testing modeling

choices. Recently, entailment metrics (FactCC;

Kryscinski et al.,2020) and contextual embedding

methods (BERTScore; Zhang et al.,2020) have sur-

faced as reasonable indicators of factuality.

3 Finding and Combating Hallucinations

It is not well understood when summarization mod-

els start to hallucinate, i.e., when they start to place

high probability on continuations that are unfaith-

ful (not entailed by the information presented in

the source document). In this work, we hypothe-

size that such moments correlate with high model

uncertainty. In other problem settings, it has been

observed that NLP models default to placing an

inappropriately large portion of probability mass

on high-frequency (with respect to the training cor-

1Pseudocode taken from Meister et al. (2020).

ROUGE-2 on XSUM has

0.17

Pearson and

0.14

Spearman

correlation (Deutsch et al.,2021;Pagnoni et al.,2021)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MutualInformationAlleviatesHallucinationsinAbstractiveSummarizationLiamvanderPoelRyanCotterellClaraMeisterlvander@ethz.chfryan.cotterell,clara.meisterg@inf.ethz.chAbstractDespitesignicantprogressinthequalityoflanguagegeneratedfromabstractivesumma-rizationmodels,thesemodelsstillexhibitthetendencytoh...

展开>> 收起<<

Mutual Information Alleviates Hallucinations in Abstractive Summarization Liam van der Poel Ryan Cotterell Clara Meister.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Mutual Information Alleviates Hallucinations in Abstractive Summarization Liam van der Poel Ryan Cotterell Clara Meister

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: