A Uniﬁed Framework for Pun Generation with Humor Principles Yufei Tian1 Divyanshu Sheth2 Nanyun Peng1 1Computer Science Department University of California Los Angeles

2025-04-30 1 0 429.63KB 9 页 10玖币

侵权投诉

A Uniﬁed Framework for Pun Generation with Humor Principles

Yufei Tian1, Divyanshu Sheth2∗

, Nanyun Peng1

1Computer Science Department, University of California, Los Angeles,

2Department of Industrial and Systems Engineering, Indian Institute of Technology, Kharagpur

{yufeit, violetpeng}@cs.ucla.edu,divyanshusheth1@gmail.com

Abstract

We propose a uniﬁed framework to generate

both homophonic and homographic puns to

resolve the split-up in existing works. Our

framework takes a theoretically motivated ap-

proach to incorporate three linguistic attributes

of puns to language models: ambiguity, dis-

tinctiveness, and surprise. Our framework con-

sists of three parts: 1) a context words/phrases

selector to promote the aforementioned humor

attributes, 2) a generation model trained on

non-pun sentences to incorporate the context

words/phrases into the generation output, and

3) a label predictor that learns the structure

of puns which is used to steer the generation

model at inference time. Evaluation results

on both homophonic and homographic puns

demonstrate the superiority of our model over

strong baselines.1

1 Introduction

Recently, computational humor theories investigat-

ing why puns are funny have shown high correla-

tions with human judgments. Kao et al. (2016) use

a probabilistic model to decompose puns into two

dimensions:

ambiguity

of meaning and

distinc-

tiveness

of viewpoints, and show that these two as-

pects combined have the strongest alignment with

human judgments (p<5

‰

). He et al. (2019) show

that ambiguity/distincitveness alone cannot capture

the whole picture, and develop an additional metric

to measure how much

surprise

is aroused when

the pun word and alternative word are ﬂipped. For

example in Figure 1, the pun word is soled and the

alternative word is sold. Seeing soled in the phrase

‘were soled at the store at half price’ instead of

sold arouses surprise in the local context but makes

sense in the global context.

Despite the success in identifying important lin-

guistic traits of successful pun sentences, how to

∗Work done when the author was interning at UCLA.

Our code is available at

https://github.com/

PlusLabNLP/Unified_PunGen

Pun word: soled Alter word: sold

Context word: boots Phrase: were sold at the store

soled

non-pun corpus

Word-level label predictor +GPT-2

learned the structure of puns

- A D1 A A - A A

The

leather

boots

he was

wearing

were heavily

D1 AD2 A - D2

abraded, and

were

soled

at the store

at half

price.

Figure 1:

An illustration of our approach. The pun word

pair (e.g. ‘soled-sold’) is the input. After retrieving a suitable

context word and a phrase, we use a pun label predictor to steer

the base GPT-2 model to produce puns. Labels D1/D2/A mean

the next word should be distinct to (supporting) the pun word,

distinct to (supporting) the alternative word, or ambiguous. A

‘-’ mark means the label predictor is less conﬁdent and thus

we do not intervene the generation process.

incorporate these aforementioned theories into the

pun generation process is still an open problem.

Although He et al. (2019) propose a retrieve-and-

edit approach to incorporate surprise, their error

analysis shows that the proposed retrieval methods

are often unsuccessful.

Moreover, existing works on pun generation

are split up in terms of generating

homographic

puns

, wherein the same written word has two or

more meanings (Mittal et al.,2022;Yu et al.,2020,

2018), and

homophonic puns

, where two words

that sound similar have different meanings (Luo

et al.,2019;He et al.,2019;Hashimoto et al.,2018).

There lacks a uniﬁed generation framework for

both types of puns.

In this work, we incorporate all three principles:

ambiguity, distinctiveness, and surprise into pun

generation, and bridge the gap between the two

pun types. We hypothesize that there is a

learn-

able structure

for puns regardless of the pun type,

and propose a uniﬁed framework by converting ho-

mographic puns to homophonic ones. Speciﬁcally,

we carefully extract from a non-pun corpus 1) a

context word that supports the meaning of the pun

word, and 2) a phrase that is both characteristic to

the alternative word and compatible with the pun

word. Next, we train a discriminator on existing

arXiv:2210.13055v1 [cs.CL] 24 Oct 2022

homophonic pun data to learn the structure of a

pun – the type of each word in the sentence, which

could be one of ‘A’ – ambiguous, ‘D1’ – distinct to

the pun word, or ‘D2’ – distinct to the alternative

word. One challenge, however, is that there are

no ground truth labels. To this end, we collect a

small amount of human annotations and boost from

weak, unsupervised models to stronger, supervised

models. At inference time, a label predictor is used

to guide a base GPT-2 model to generate puns. At

each generation step, we re-score the tokens gener-

ated by the base language model according to the

predicted type, except for the case when the label

predictor’s conﬁdence is under a set threshold. Our

model outperforms existing baselines for both pun

types.

2 Related Works

Linguistic traits of puns.

Kao et al. (2016) de-

compose puns into two dimensions — ambiguity

of meaning and distinctiveness of viewpoints, and

show that that ambiguity is useful in distinguishing

non-puns from puns, while distinctiveness is useful

when spotting good and funny puns from bad or

boring non-puns. To the best of our knowledge,

we are the ﬁrst to formally incorporate the famous

ambiguity-distinctiveness principle to guide pun

generation. In addition, He et al. (2019) propose

the local-global surprisal principle to measure the

humorous effect aroused when a word appears un-

expectedly in the local context but makes sense

given the global context, based on which we im-

prove the way surprise is introduced in generation.

Pun generation.

Existing works on pun gener-

ation often rely on naive intuitions of semantic

ambivalence. For example, Yu et al. (2018) and

Luo et al. (2019) promote the ambivalence of the

pun word via a constrained language model and re-

inforcement learning; others ﬁnd related words to

support semantic ambiguity (Yu et al.,2020;Mittal

et al.,2022). However, these systems lack seri-

ous theoretical backbones and therefore none could

evaluate their generated results with regards to the

proposed intuitions. What’s more, the nature of

‘ambivalence’ alone leads to generic/boring word

choice and short outputs. By incorporating distinc-

tiveness and surprise, we ensure that the generated

puns are informative and interesting.

One reason that previous works leverage those

simple intuitions to generate puns (He et al.,2019;

Yu and Wan,2019;Yu et al.,2020) is that the small

corpora size (Miller et al.,2017;Sun et al.,2022a)

makes it impractical to train generation models

end-to-end using human written puns. We hence

propose to learn the structure of puns instead of

the actual texts, which requires far less data to train

on. Finally, all previous works (except a concurrent

one (Sun et al.,2022b)) can only generate either

homographic puns or homophonic puns. Leverag-

ing the shared structure of puns regardless of the

pun type, our model can generate both pun types.

3 Methodology

The input to our system is a pun word-alternative

word pair (

, e.g., soled-sold), and the target

output is a high-quality pun sentence that contains

, e.g., ‘The leather boots he was wearing were

heavily abraded, and were soled at the store at

half price.’ In this section, we ﬁrst describe the

three components to generate homophonic puns

as shown in Figure 1: a context word and phrase

selector, a label predictor and the procedure of cu-

rating training signals, and the generation module

in Section 3.1 to 3.3. Then, we migrate the whole

system to homographic puns in Section 3.4.

3.1 Obtaining Context Words and Phrases

We retrieve and select two things: a context word

that supports the meaning of the pun word, and a

phrase that is both characteristic to the alternative

word and compatible with the pun word.

Inspired by He et al. (2019), given a pun-

alternative word pair (

pw −aw

), we look for an

ideal phrase that contains

and replace it with

to arouse surprise. To this end, we ﬁrst extract

multiple (

=20) phrases that contain

from

a large non-pun corpus consisting of 20,000 sen-

tences from Wikipedia and Gutenberg BookCorpus

(Lebert,2009), and rank the phrases by how well

they exhibit the semantics of the pun pair. Speciﬁ-

cally, we ﬁrst replace

with a ‘<mask>’ token,

and run RoBERTa-Large (Liu et al.,2019) to obtain

the probability of

in the masked position. We

remove the less probable half, ﬁltering out those

that are less characteristic of

, as shown in Table

1. Next, we conduct a similar mask inﬁlling proce-

dure for

, and select the middle-ranking phrase

to avoid it being either too general (e.g., ‘a new taxi

was created’) or too incompatible (e.g., ‘an export

taxi on agricultural products’). These two rankings

ensure the ﬁnal selected phrase arouses surprisal

when people see

instead of

, but also still

ﬁnd it reasonable.

For obtaining the context words, our idea is sim-

ilar to that proposed by (Mittal et al.,2022). We

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AUniedFrameworkforPunGenerationwithHumorPrinciplesYufeiTian1,DivyanshuSheth2,NanyunPeng11ComputerScienceDepartment,UniversityofCalifornia,LosAngeles,2DepartmentofIndustrialandSystemsEngineering,IndianInstituteofTechnology,Kharagpur{yufeit,violetpeng}@cs.ucla.edu,divyanshusheth1@gmail.comAbstractWe...

展开>> 收起<<

A Uniﬁed Framework for Pun Generation with Humor Principles Yufei Tian1 Divyanshu Sheth2 Nanyun Peng1 1Computer Science Department University of California Los Angeles.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

A Uniﬁed Framework for Pun Generation with Humor Principles Yufei Tian1 Divyanshu Sheth2 Nanyun Peng1 1Computer Science Department University of California Los Angeles

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: