Dense Paraphrasing for Textual Enrichment Jingxuan Tu Kyeongmin Rim Eben Holderness James Pustejovsky Department of Computer Science_2

2025-05-06 0 0 1.71MB 18 页 10玖币

侵权投诉

Dense Paraphrasing for Textual Enrichment

Jingxuan Tu, Kyeongmin Rim, Eben Holderness, James Pustejovsky

Department of Computer Science

Brandeis University

Waltham, Massachusetts

{jxtu,krim,egh,jamesp}@brandeis.edu

Abstract

Understanding inferences and answering

questions from text requires more than

merely recovering surface arguments, ad-

juncts, or strings associated with the query

terms. As humans, we interpret sentences

as contextualized components of a narrative

or discourse, by both ﬁlling in missing in-

formation, and reasoning about event con-

sequences. In this paper, we deﬁne the pro-

cess of rewriting a textual expression (lex-

eme or phrase) such that it reduces ambiguity

while also making explicit the underlying se-

mantics that is not (necessarily) expressed in

the economy of sentence structure as Dense

Paraphrasing (DP). We build the ﬁrst com-

plete DP dataset, provide the scope and de-

sign of the annotation task, and present re-

sults demonstrating how this DP process can

enrich a source text to improve inferencing

and Question Answering (QA) task perfor-

mance. The data and the source code will be

publicly available.

1 Introduction

Two of the most important components of under-

standing natural language involve recognizing that

many different textual expressions can correspond

to the same meaning, and detecting those aspects

of meaning that are not present in the surface form

of an utterance or narrative. Together, these in-

volve broadly three kinds of interpretive processes:

(i) recognizing the diverse variability in linguistic

forms that can be associated with the same underly-

ing semantic representation (paraphrases); (ii) (e.g.,

stir vigorously); and (iii) interpreting or computing

the dynamic consequences of actions and events in

the text (e.g., slicing an onion brings about onion

slices).

The ﬁrst of these, the problem of paraphrasing,

has been addressed computationally since the early

days of natural language processing (NLP). The

second and third dimensions of sentence meaning

mentioned above, however, are more difﬁcult to

model with current ML approaches, which rely

heavily on explicit textual strings to model seman-

tic associations between the elements in the in-

put. Many question answering systems, for exam-

ple, rely on such syntagmatic forms in the train-

ing data for modeling potential associations that

contribute to completion or generation task per-

formance. Hence, if predicates or arguments are

missing, implied, or interpreted from context, there

is nothing to encode, and consequently little to

decode as output, as well. Consider the follow-

ing example from the traditional paraphrasing task.

The text difference between the input and output

only comes from a lexical substitution, rather than

the rephrasing or addition of hidden arguments.

(1) Paraphrasing:

Chop onions, saute until browned. −→

Cut onions, saute until done.

To solve this problem, some recent attempts have

been made to enrich surface sentence forms that are

missing information through “decontextualization”

procedures that textually supply information which

would make the sentence interpretable out of its

local context (Choi et al.,2021;Elazar et al.,2021;

Wu et al.,2021). Choi (2021) formally deﬁnes

decontextualization as:

Deﬁnition 1.1. Decontextualization

: Given a

sentence-context pair

(S, C)

, a sentence

is a

valid decontextualization of s if: (1) the sentence

is interpretable in the empty context; and (2) the

truth-conditional meaning of

in the empty con-

text is the same as the truth-conditional meaning of

Sin context C.

The decontextualization task focuses on enrich-

ing text through anaphora resolution and knowl-

edge base augmentation, which works well on ar-

guments or concepts that can be linked back to

arXiv:2210.11563v1 [cs.CL] 20 Oct 2022

existing knowledge sources, such as Wikipedia.

Consider the following example of the decontextu-

alization task. It is able to decontextualize Barilla

sauce in (2a), but does not reintroduce any seman-

tically hidden arguments from the context in (2b),

making inferences over such sentences difﬁcult or

impossible.

(2) Decontextualization:

a. Add Barilla sauce, salt and red pepper

ﬂakes. −→

Add Barilla sauce,

the tomato sauce,

salt and

red pepper ﬂakes.

b. Simmer 2 minutes over medium heat. −→

Simmer 2 minutes over medium heat.

In this paper, we argue that the problems of para-

phrasing and decontextualizing are closely related,

and part of a richer process of what we call Dense

Paraphrasing. This combines the textual variability

of an expression’s meaning (paraphrase) with the

ampliﬁcation or enrichment of meaning associated

with an expression (decontextualization).

While a paraphrase is typically deﬁned as a rela-

tion between two expressions that convey the same

meaning (Bhagat and Hovy,2013), it has also been

used to clarify meaning through verbal, nominal, or

structural restatements that preserve (and enhance)

meaning (Smaby,1971;Kahane,1984;Mel’cuk,

1995;Mel’ ˇ

Cuk,2012), in particular the notion of

“entailed paraphrase” (Culicover,1968): (author,

person who writes), (sicken,to make ill), (strong,

potent (of tea)).

This is clearly related to recent efforts at decon-

textualizing linguistic expressions with “contextual

enrichments" (Choi et al.,2021). What these ap-

proaches do not focus on, however, is the notion of

enrichment of the expression through both its lexi-

cal semantics and its dynamic contribution to the

text in the narrative. We deﬁne a Dense Paraphrase

(DP) as follows:

Deﬁnition 1.2. Dense Paraphrasing

: Given the

pair,

(S, P )

, where

is a source expression, and

is an expression, we say

is a valid Dense Para-

phrase of

if:

is an expression (lexeme, phrase,

sentence) that eliminates any contextual ambigu-

ity that may be present in

, but that also makes

explicit any underlying semantics that is not other-

wise expressed in the economy of sentence struc-

ture, e.g., default or hidden arguments, dropped

objects or adjuncts.

is both meaning preserv-

ing (consistent) and ampliative (informative) with

respect to S.

The following shows the DPs of the sentences

from examples (1) and (2). Compared to the afore-

mentioned tasks, a DP aims to recover semantically

hidden arguments through: (1) a broader view of

the context of the text; and (2) commonsense or

best educated guesses from humans (i.e., text spans

with underlines from the example).

(3) Chop onions, saute until browned. −→

Chop onions

on a cutting board with a knife

to get

chopped onions

, saute

chopped onions

on a pan with a spatula

until browned, result-

ing in sauted chopped onions.

——————————————————

Add Barilla sauce, salt and red pepper ﬂakes to

the saucepan. Simmer 2 minutes over medium

heat. −→

Add Barilla sauce, salt and red pepper ﬂakes

to the saucepan

by hand to get sauce mixture

Simmer the

sauce mixture

2 minutes

in the

saucepan

over medium heat to get

simmered

sauce mixture.

We argue that our work can potentially help and

complement these generation tasks by enriching

the source text with information that is not on the

surface, by either additional text strings or vec-

tor representations. To show the usage of DP, we

evaluate our method through QA tasks on dense-

paraphrased questions.

In the remainder of the paper, we ﬁrst review

related work and background (§2), and give more

detailed deﬁnitions of the DP schema (§3). We then

introduce a dataset we have created to support our

implementation of the DP operation (§4), immedi-

ately followed by the details of how we collected

and annotated this dataset (§5). §6provides details

of experiments we conducted to validate the util-

ity of the proposed methodology, along with their

results. Then conclude our work in the ﬁnal (§7).

2 Background

There is a long history in linguistics, dating back

to the early 1960s, of modeling linguistic syntag-

matic surface form variation in terms of transfor-

mations or sets of constructional variants (Harris,

1954,1957) (Hi˙

z,1964). When these transforma-

tions are viewed “derivationally", i.e., as an ordered

application of rules over an underlying form, the

resulting theory is in the family of generative gram-

mars (Chomsky,1957;Bach,1964). If they are

Passage: Peel and cut apples into wedges.Press apple wedges partly into batter.Combine sugar and

cinnamon.Sprinkle over apple.Bake at 425 degF for 25 to 30 minutes.

Dense Paraphrased (DP’ed) Passage:

Using peeler, peel apples, resulting in peeled apples; and using knife on cutting board, cut peeled

apples into peeled wedges.

Using hands, press peeled apple wedges partly into batter in the cake pan.

Combine sugar and cinnamon in a bowl, resulting in cinnamon sugar.

Sprinkle cinnamon sugar over peeled apple wedges in batter in cake pan, resulting in appelkoek.

In oven, bake appelkoek at 425 degF for 25 to 30 minutes, resulting in baked appelkoek.

Table 1: Example recipe passage. Color-coded text spans represent locations of cooking events in the

input text where Dense Paraphrases (DPs) are generated to enrich local context. Underlined text shows a

chain of coreferential entities for the ingredient “apple”.

seen as undifferentiated choices over surface con-

structional forms of an expression, the resulting the-

ory can be called a paraphrase grammar (Hi˙

z,1964;

Smaby,1971;Culicover,1968). Formally, a para-

phrase is a relation between two lexical, phrasal, or

sentential expressions, Eiand Ej, where meaning

is preserved (Smaby,1971).

For NLP uses, paraphrasing has been a major

part of machine translation and summarization

system performance (Culicover,1968;Goldman,

1977;Muraki,1982;Boyer and Lapalme,1985;

McKeown,1983;Barzilay and Elhadad,1999;Bha-

gat and Hovy,2013). In fact, statistical and neural

paraphrasing is a robust and richly evaluated com-

ponent of many benchmarked tasks, notably MT

and summarization (Weston et al.,2021), as well as

Question Answering (Fader et al.,2013) and seman-

tic parsing (Berant and Liang,2014). To this end,

signiﬁcant efforts have gone towards the collection

and compilation of paraphrase datasets for training

and evaluation (Dolan and Brockett,2005;Ganitke-

vitch et al.,2013;Ganitkevitch and Callison-Burch,

2014;Pavlick et al.,2015;Williams et al.,2017).

In addition to above meaning-preserving para-

phrase strategies, there are several directions cur-

rently that use strategies of “decontextualization”

or “enrichment” of a textual sequence, whereby

missing, elliptical, or underspeciﬁed material is

re-inserted into the expression. The original and

target sentences are compared and judged by an

evaluation as a text generation or completion task

(Choi et al.,2021;Elazar et al.,2021).

Enrichment of VerbNet predicates can be seen

as an early attempt to provide a kind of Dense

Paraphrasing for the verb’s meaning. In Im and

Pustejovsky (2009,2010), the basic logic of Gen-

erative Lexicon’s subevent structure was applied

to VerbNet classes, to enrich the event repre-

sentation for inference. The VerbNet classes

were associated with event frames within an

Event Structure Lexicon (ESL) (Im and Puste-

jovsky,2010), encoding the subevent structure of

the predicate. If the textual form for the verb

is replaced with the subeventual description it-

self, classes such as

change_of_location

and

change_of_possession

can help encode and

describe event dynamics in the text, as shown in

(Brown et al.,2018;Dhole and Manning,2021;

Brown et al.,2022). For example, the VerbNet

entry drive is enriched with the ESL subevent struc-

ture below:

(4) drive in John drove to Boston

se1: pre-state: not_located_in (john,boston)

se2: process: driving (john)

se3: post-state: located_in (john,boston)

In the remainder of the paper, such techniques will

be utilized as part of our Dense Paraphrasing strat-

egy to enrich the surface text available for language

modeling algorithms.

3 Method: Dense Paraphrasing

In this section, we detail the procedure involved

in creating DPs for a text. Compared to decon-

textualization, DP can be seen as similar, but is a

much broader method for creating sets of seman-

tically equivalent or “enriched consistent" expres-

sions, that can be exploited for either human or

machine consumption.

Unlike traditional paraphrases that are evaluated

in terms of how faithful and complete they are,

while preserving the literal interpretation of the

source, the goal of our task is to generate dense

paraphrases that can be merged with the concept

of semantic enrichment, to give rise to a set of

paraphrases of semantically enriched and decontex-

tualized expressions. We distinguish between two

contexts of use for a paraphrase:

Deﬁnition 3.1. Human Readable Paraphrase

(HRP)

: the redescription of the source expression,

, generated as a paraphrase of

, is intended

to be read, viewed, or heard by a human audience.

Context, style, genre, register, and voice may dic-

tate nuanced variations in the resulting form of the

paraphrase;

Deﬁnition 3.2. Machine Readable Paraphrase

(MRP)

: the source expression,

, is enriched with

descriptive content and contextualized information

that turns implicit content into explicit textual ex-

pressions. The output of MRP is logically con-

sumed by a downstream model, such as a question-

answering system, that can utilize the richer local

environment for improved accuracy on a variety of

reasoning tasks.

Consider the following example of the DP of the

original recipe sentence. Table 1shows a dense

paraphrased passage from this data. The original

text and corresponding dense paraphrased text are

associated with the same color. Both HRP and

MRP formsd from the same sentence are illustrated.

The information that is encoded in both paraphrases

is identical. HRP includes the insertion of addi-

tional prepositions and the proper ordering of tex-

tual components, while MRP includes metadata

content to structure the arguments and relations.

(5) Chop onions, ... −→

HRP: Chop onions on a cutting board with a

knife to get chopped onions

MRP: Chop {

TOOL

:knife #

HABITAT

:cutting

board #

OUTCOME

:chopped onions} onions

{INGRE_OF:chop}

3.1 Dense Paraphrasing Procedure

In this section, we describe the mechanisms in-

volved in creating a DP from a source text. In this

work we will focus on the MRP, since our present

goal is creating DPs that can be used in the ser-

vice of NLP applications. Speciﬁcally, we adopt

a template-based method along with heuristics to:

(1) generate dense-paraphrases that account for hid-

den entities and entity subevent structure; and (2)

convert them to quasi-grammatical text formats for

machine consumption.

We provide the source narrative with a dynamic

DP of the surface text, which both decontextual-

izes the expression (Choi et al.,2021;Elazar et al.,

2021), but also enriches the textual description of

both events and participants to reﬂect the changes

in the object due to the events. DP involves iden-

tifying conventional coreference chains where the

entities from the chain are identical. Importantly,

this procedure also includes additional textual de-

scriptions involving: (1) recovering “hidden” ar-

guments; and (2) the results of subevent decom-

position (Pustejovsky,1995;Im and Pustejovsky,

2010), which create the coreference relation be-

tween the entity and its hidden or transformed men-

tion elsewhere (e.g. apples

→

apple wedges

→

applekoek).

3.1.1 Recovering “hidden” Arguments

We deﬁne a “hidden” argument to a predicate as an

event participant that is not present in the surface

form of the text. Given this, we distinguish two

subtypes of hidden arguments: 1

•Drop argument

: A drop argument is an argu-

ment to a predicate that has been elided or left

unexpressed in the syntax. Such elisions oc-

cur when the antecedent has been mentioned

in a previous sentence and can be recovered

from the context in the document.

•Shadow argument

: A shadow argument is

semantically incorporated in the meaning of

the event predicate itself; e.g., an implicit tool

or ingredient that is not mentioned but pre-

supposed (Pustejovsky,1995;Johnson et al.,

2002).

We manually annotate our data to identify all

the hidden arguments (both syntactic and semantic)

associated with an event predicate. This effectively

“saturates" the lexical frame (Fillmore,1985) by

supplying those frame elements needed to perform

richer inferential tasks.

3.1.2 Recovering Entity Properties from

Event Structure

As mentioned above, ESL represents an event as

having three parts:

begin (Be)

inside (Ie)

, and

end

O’Gorman et al. (2018a) uses implicit role to cover drop

arguments in AMR. However, shadow arguments do not ap-

pear to fall under their category of implicit roles. To our

knowledge, this work is the ﬁrst to annotate the information

associated with shadow arguments in verbal constructions.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DenseParaphrasingforTextualEnrichmentJingxuanTu,KyeongminRim,EbenHolderness,JamesPustejovskyDepartmentofComputerScienceBrandeisUniversityWaltham,Massachusetts{jxtu,krim,egh,jamesp}@brandeis.eduAbstractUnderstandinginferencesandansweringquestionsfromtextrequiresmorethanmerelyrecoveringsurfaceargument...

展开>> 收起<<

Dense Paraphrasing for Textual Enrichment Jingxuan Tu Kyeongmin Rim Eben Holderness James Pustejovsky Department of Computer Science_2.pdf

共18页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Dense Paraphrasing for Textual Enrichment Jingxuan Tu Kyeongmin Rim Eben Holderness James Pustejovsky Department of Computer Science_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: