Why Do You Feel This Way Summarizing Triggers of Emotions in Social Media Posts Hongli Zhan1Tiberiu Sosea2Cornelia Caragea2Junyi Jessy Li1

2025-05-06 0 0 2.86MB 19 页 10玖币

侵权投诉

Why Do You Feel This Way?

Summarizing Triggers of Emotions in Social Media Posts

Hongli Zhan∗1Tiberiu Sosea∗2Cornelia Caragea2Junyi Jessy Li1

1Department of Linguistics, The University of Texas at Austin

2Department of Computer Science, University of Illinois Chicago

{honglizhan,jessy}@utexas.edu {tsosea2,cornelia}@uic.edu

Abstract

Crises such as the COVID-19 pandemic con-

tinuously threaten our world and emotionally

affect billions of people worldwide in distinct

ways. Understanding the triggers leading to

people’s emotions is of crucial importance. So-

cial media posts can be a good source of such

analysis, yet these texts tend to be charged

with multiple emotions, with triggers scatter-

ing across multiple sentences. This paper takes

a novel angle, namely, emotion detection and

trigger summarization, aiming to both detect

perceived emotions in text, and summarize

events and their appraisals that trigger each

emotion. To support this goal, we introduce

COVIDET (Emotions and their Triggers dur-

ing Covid-19), a dataset of ~1,900 English

Reddit posts related to COVID-19, which con-

tains manual annotations of perceived emo-

tions and abstractive summaries of their trig-

gers described in the post. We develop strong

baselines to jointly detect emotions and sum-

marize emotion triggers. Our analyses show

that COVIDET presents new challenges in

emotion-speciﬁc summarization, as well as

multi-emotion detection in long social media

posts.

1 Introduction

Large-scale crises such as the COVID-19 pandemic

continuously cause emotional turmoil worldwide.

People are emotionally affected in different ways,

e.g., online education has led to mental health is-

sues among students (Akpınar et al.,2021) as well

as parents (Cui et al.,2021); lock-down policies are

protective for the vulnerable (Flaxman et al.,2020;

Hsiang et al.,2020) while economically disastrous

for many (Odii et al.,2021). Emotion analysis —

both detecting emotion and understanding what

triggers the emotion — brings invaluable insights

both practically (e.g., for ﬁrst responders, coun-

selors, etc) and in scientiﬁc research (Arora et al.,

*Hongli Zhan and Tiberiu Sosea contributed equally.

Reddit Post

1: My sibling is 19 and she constantly goes places with her friends and to there

houses and its honestly stressing me out.

2: Our grandfather lives with us and he has dementia along with other health issues

and my mom has diabetes and heart problems and I have autoimmune diseases

& chronic health issues.

3: She also has asthma.

4: Its stressing me out because despite this she seems to not care about how badly

it would affect all of us if we were to get the virus.

5: And sadly I feel like its not much I can do she literally doesn’t respect my mom

and though I’m older she doesn’t respect me either.

6: Its so frustrating.

Emotions and Abstractive Summaries of Triggers

Emotion: anger

Abstractive Summary of Trigger: My sister having absolutely no regard for any of

our family’s health coupled with the fact that I can’t do anything about it is so

aggravating to me.

Emotion: fear

Abstractive Summary of Trigger: My sibling, who, in spite of our family’s myriad

of issues that all make us high-risk people, continuously goes out and about, which

makes her likely to get infected. I am scared for all of us right now.

Figure 1: An example from COVIDET, with perceived

emotion(s) identiﬁed and their trigger(s) summarized.

2021;Uban et al.,2021).

While emotion detection (typically formulated

as a classiﬁcation task among standard emotion tax-

onomies) is a well-established task (e.g., Mihalcea

and Strapparava (2012); Wang et al. (2012); Abdul-

Mageed and Ungar (2017); Khanpour and Caragea

(2018); Demszky et al. (2020) and in crises con-

texts (Desai et al.,2020;Sosea et al.,2022)), fewer

have studied

what leads to these emotions

in the

scope of the text concerned in a data-driven man-

ner. Xia and Ding (2019) adopt an extraction setup

to identify emotion “causes” that is limited to the

clause level, where only one (explicitly expressed)

emotion and one cause are associated. This does

not generalize to long, spontaneous social media

posts that are emotionally charged. Illustrated in

Figure 1, distinct emotions are triggered by differ-

ent events across multiple sentences.

Additionally, how these events are subjectively

evaluated, interpreted or appraised, e.g., “I can’t

do anything about it” in the ﬁrst example of Fig-

ure 1, also contribute to the emotion (Smith and

Ellsworth,1985;Ellsworth and Scherer,2003).

The fact that different individuals may have distinct

appraisals towards the same event (Moors et al.,

arXiv:2210.12531v1 [cs.CL] 22 Oct 2022

2013) further highlights the challenging nature of

understanding what triggers an emotion.

In this work we take a novel view, and formulate

emotion-trigger detection as an abstractive summa-

rization task that synthesizes a natural language

description of the events and their appraisals that

trigger a particular emotion. We frame our work

as emotion detection and trigger summarization

(Figure 1), which entails both detecting perceived

emotions in text, and summarizing triggers for each

emotion.

We present COVIDET (

motions and their

riggers during

Covid

-19), a new dataset sourced

from

1,883

English Reddit posts about the COVID-

19 pandemic. Each post is annotated with 7 ﬁne-

grained emotion labels; for each emotion, annota-

tors provided a concise, abstractive summary de-

scribing the triggers of the emotion. The triggers

are further validated in a separate stage. COVIDET

spans from June 2021 to January 2022, captur-

ing various signiﬁcant events as well as how they

were emotionally appraised during the pandemic.

Compared to prior emotion studies that consider

only sentence-level texts (Sosea and Caragea,2020;

Demszky et al.,2020) or (short) tweets (Sosea et al.,

2022;Abdul-Mageed and Ungar,2017), COVIDET

is challenging as it contains signiﬁcantly longer

texts. We showcase examples of COVIDET in Ap-

pendix §A.

Analyses of COVIDET reveal that negative emo-

tions such as fear and anger are prevalent. These

emotions co-occur most frequently with anticipa-

tion, which consistently rise after the Omicron sub-

variant became more dominant with fear dropping.

Topic modeling over the trigger summaries points

to irritations toward those who don’t mask or get

vaccinated, and positivity towards the vaccines.

Using COVIDET, we benchmark models for

emotion detection and emotion-trigger summariza-

tion. We employ both separate emotion detection

and trigger summarization models, as well as joint

models that we designed to simultaneously detect

emotions and generate trigger summaries. Our ex-

periments showcase the distinct nature of our task,

emphasizing that COVIDET is vital to training re-

liable emotion detection and trigger summariza-

tion approaches in a Covid-19 context. COVIDET

bears various unique characteristics, ranging from

its long sequences and invaluable context to the na-

ture of the task itself. Therefore, general emotion

detection or summarization models unsurprisingly

lag behind in performance compared to our meth-

ods. Moreover, human evaluation of the generated

trigger summaries tailored for emotion-trigger sum-

marization indicates that our models are effective

in capturing the underlying triggers of the post.

We release COVIDET and our code at

https:

//github.com/honglizhan/CovidET.

2 Related Work

Summarization.

Recent pre-trained models led

to substantial progress in single document summa-

rization. In the case of abstractive summarization,

encoder-decoder transformer models are used to

synthesize a concise description of the most salient

concepts in the input (Lewis et al.,2020;Zhang

et al.,2020). Signiﬁcant efforts in summarization

focus on news because of the availability of large

datasets such as CNN/DailyMail (Hermann et al.,

2015) and XSum (Narayan et al.,2018); in the do-

main of social media, TL;DR sentences has been

mined in Reddit to serve as summaries and train

models (Völske et al.,2017;Kim et al.,2019).

However, generic summaries tend not to be infor-

mative if users are concerned with speciﬁc emo-

tions expressed.

In this sense our setup ﬁts into settings where

only a certain part of the content is of interest to the

user. We could view our task as answering a query,

“Why does the writer feel [emotion]?”. However,

such queries are more general than query-based

summarization (Daumé III and Marcu,2006;Otter-

bacher et al.,2009;Schilder and Kondadadi,2008;

Nema et al.,2017;Baumel et al.,2018;Laskar

et al.,2020;Su et al.,2021;Zhong et al.,2021),

where queries tend to be more document-speciﬁc.

Perhaps a closer task is opinion summarization, or

aspect-based summarization more generally. In

opinion summarization, models need to summarize

affect/opinions about a certain aspect of a service or

product (Popescu and Etzioni,2005;Angelidis and

Lapata,2018;Huy Tien et al.,2019;Suhara et al.,

2020;Angelidis et al.,2021;Amplayo and Lapata,

2021); on the contrary, our setup entails identify-

ing the emotions and summarizing the events and

how they were made sense of with respect to each

emotion. In aspect-based summarization, existing

work has explored summarizing with respect to pre-

designated aspects of certain news (Frermann and

Klementiev,2019;Ahuja et al.,2022), and entities

mentioned in text (Maddela et al.,2022).

Emotion Cause Extraction.

Emotion Cause Ex-

traction (ECE) is a task that aims to extract the

events triggering a particular emotion (Khunteta

and Singh,2021). ECE was ﬁrst introduced by Lee

et al. (2010), where they deﬁned the task as extract-

ing word-level causes to the given emotion in text.

Chen et al. (2010) and Gui et al. (2016) expanded

the task to clause-level cause detection; Xia and

Ding (2019) aimed to removed the constraint that

emotions must be human-annotated before conduct-

ing automatic cause extraction, and thus proposed

Emotion-Cause Pair Extraction (ECPE) aiming to

extract potential pairs of emotions and causes in

a document. Most of the datasets are in Chinese,

in either micro-blog or news domains (Gao et al.,

2015;Gui et al.,2016;Gao et al.,2017).

In contrast, we study a more generalized no-

tion of triggers of an emotion where readers are

asked to actively appraise and interpret the emo-

tions together with their stimuli in the document,

rather than solely identifying the events behind

each emotion. We use abstractive summarization

to handle triggers, which can better synthesize inter-

connected complex events and abstract concepts,

as well as making the output contextually indepen-

dent.

3 Dataset Construction

We present COVIDET, a novel dataset from En-

glish Reddit posts that is manually annotated with

emotions and summaries of their triggers. This sec-

tion discusses the data creation process; in §4, we

discuss inter-annotator agreement and our human

veriﬁcation process.

3.1 Selecting & Curating Reddit Posts

We gather posts from

r/COVID19_support1

. We

select it as the source of our data because of its rich

personal narration: rather than COVID-19 news

snippets, this subreddit is targeted for people seek-

ing any community support during the pandemic.

We randomly sample posts before (from Jun 23,

2021 to Oct 1, 2021) and after (from Dec 1, 2021

to Jan 25, 2022) Omicron, a COVID-19 variant that

emerged during December 2021.

We restrict posts to be between

400

tokens

long (punctuation excluded); this allows us to have

posts that are long enough, but still manageable

for crowdsourcing tasks. Close scrutiny shows that

the posts in COVIDET center around 100 tokens in

1https://www.reddit.com/r/COVID19_support/

50 100 150 200 250 300 350 400

100

150

200

Count

Figure 2: Distribution of the length of posts in

COVIDET.

length; the distribution of the length of the posts

is given in Figure 2. The average length of posts

in COVIDET is

156.4

tokens (std.dev =

83.3

). We

mask web links with an

[url]

token and do not

provide the metadata to our annotators. Note that 6

posts have length under 50 tokens: this is because

we performed

[url]

masking after length ﬁltering

when collecting the source data. Details of the full

preprocessing procedure are provided in Appendix

§B.

3.2 Annotation Task

Instructions.

Annotators are ﬁrst asked to anno-

tate Plutchik basic emotions (Plutchik,2001) they

perceive: anger, anticipation, joy, trust, fear, sad-

ness, and disgust.

Multiple selection is allowed,

and we also provide a none of the above option in

case no emotion is perceived.

Once the annotators select an emotion, they are

asked to summarize the trigger(s) to their perceived

emotions, speciﬁcally an abstractive summary in

their own words, in the author’s voice. The sum-

maries should contain trigger(s) to the emotion

rather than just reﬂecting the emotion itself. We

provide the detailed instructions to our annotation

task in Appendix §C.

Annotators

We recruit two different groups of

annotators. The ﬁrst group consists of trained turk-

ers from Amazon Mechanical Turk. The workers’

locale is the US, and they have completed

500

HITs with an acceptance rate

≥95

%. This group

contributes to COVIDET’s training and validation

sets. The second group consists of 2 linguistic un-

dergraduate students, who contributes to the test set.

To ensure the quality of COVIDET, both groups

After annotation, we found very little surprise in the data

(59 in total), thus we leave out surprise for this work.

of annotators are trained and qualiﬁed in a pre-

qualiﬁcation process. We also ask them to revise

their work when needed during annotation.

Pre-Annotation Training

We trained the anno-

tators before they annotate COVIDET. We set up a

qualiﬁcation task on the Amazon Mechanical Turk.

The qualiﬁcation task involves 3 posts, and annota-

tors are required to complete the qualiﬁcation task.

Through manually examining the annotators’ work

on the qualiﬁcation task and comparing the annota-

tions to the gold annotations we develop, we ﬁlter

high-quality annotators and give them the access to

our annotation task. We also provide feedback to

their annotations. The turkers are paid at least $10

per hour. To ensure this goal is reached, we keep

track of their working time on the backstage and

give out bonuses accordingly when needed.

Annotation Revisions

During the process of the

annotation on COVIDET, we regularly review the

annotations and give feedback accordingly. When

needed, we send the annotations back to the anno-

tator along with the original post, and ask them to

revise their work based on our suggestions. Note

that the annotator is responsible for the revision of

their own work only.

3.3 Benchmark Dataset

We annotated

1,485

posts on the Amazon Mechan-

ical Turk, each annotated by two independent work-

ers. Since the neutral class is very infrequent, we

remove it from our experiments. To facilitate our

experiments, we split the examples into

1,200

ex-

amples for training and

285

examples for valida-

tion. Our test set—which is annotated by linguistic

undergraduates—contains 398 examples.

If at least one annotator labels a post with an

emotion

, then we include emotion

as an emo-

tion label. In cases where both annotators assign

an emotion

to a post, we consider the trigger sum-

maries as two separate training examples for the

trigger summarization task. In cases where a post

has two different trigger summaries in the valida-

tion or the test set, we consider them as multiple

references when computing our metrics.

4 Agreement and Validation

To account for the quality of COVIDET, we mea-

sure the inter-annotator agreement in emotions

(§4.1) and triggers (§4.2). The annotations are

further validated through human inspection in §4.3.

0 200 400 600 800 1000 1200

Number of Examples

none

trust

joy

disgust

sadness

anger

fear

anticipation

0.4

0.5

0.6

0.7

0.8

Inter-annotator PEA Score

Figure 3: Emotion distribution of COVIDET, ranked

by the number of examples. Colors indicate the inter-

annotator agreement measured by the PEA score.

Results reveal that annotators tend to agree with

each other in emotions whilst using varied vocabu-

laries when summarizing the triggers.

4.1 Agreement in Emotions

Percentage Overlap.

For each example in

COVIDET, we measure the number of emotions in

which both annotators agree upon. Results show

that in 81.4% of the examples in COVIDET, both

annotators agree on at least 1 emotion label; in

26.6% of the examples, both annotators agree on

at least 2 emotion labels.

PEA Score.

To account for distances between

emotions (e.g., disgust is further away from joy

than from anger), we report the Plutchik Emotion

Agreement (PEA) metric (Desai et al.,2020) for

the inter-annotator agreement of emotions anno-

tated in COVIDET. We ﬁrst report the average PEA

score among annotators weighted by their numbers

of annotations, which is 0.8 for the training and

validation sets combined, and 0.821 for the test

set (0.804 for all three combined). These numbers

indicate high agreement (Desai et al.,2020).

Figure 3shows per-emotion PEA scores, along

with the frequency of each emotion. All emotions

have high agreement; the highest are among fear

and anger, with the average PEA scores at around

0.85; the lowest is trust, with the average PEA

score at around 0.74.

Finally, to calculate agreement between students

and crowd workers, we randomly sample 208 exam-

ples from the training set and ask the linguistic un-

dergraduate students to annotate them from scratch.

We assign one student per example for validation.

The average PEA score between crowd workers

and linguistics students is 0.832, suggesting high

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

WhyDoYouFeelThisWay?SummarizingTriggersofEmotionsinSocialMediaPostsHongliZhan1TiberiuSosea2CorneliaCaragea2JunyiJessyLi11DepartmentofLinguistics,TheUniversityofTexasatAustin2DepartmentofComputerScience,UniversityofIllinoisChicago{honglizhan,jessy}@utexas.edu{tsosea2,cornelia}@uic.eduAbstractCrises...

展开>> 收起<<

Why Do You Feel This Way Summarizing Triggers of Emotions in Social Media Posts Hongli Zhan1Tiberiu Sosea2Cornelia Caragea2Junyi Jessy Li1.pdf

共19页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Why Do You Feel This Way Summarizing Triggers of Emotions in Social Media Posts Hongli Zhan1Tiberiu Sosea2Cornelia Caragea2Junyi Jessy Li1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: