Why Do You Feel This Way Summarizing Triggers of Emotions in Social Media Posts Hongli Zhan1Tiberiu Sosea2Cornelia Caragea2Junyi Jessy Li1

2025-05-06 0 0 2.86MB 19 页 10玖币
侵权投诉
Why Do You Feel This Way?
Summarizing Triggers of Emotions in Social Media Posts
Hongli Zhan∗1Tiberiu Sosea∗2Cornelia Caragea2Junyi Jessy Li1
1Department of Linguistics, The University of Texas at Austin
2Department of Computer Science, University of Illinois Chicago
{honglizhan,jessy}@utexas.edu {tsosea2,cornelia}@uic.edu
Abstract
Crises such as the COVID-19 pandemic con-
tinuously threaten our world and emotionally
affect billions of people worldwide in distinct
ways. Understanding the triggers leading to
people’s emotions is of crucial importance. So-
cial media posts can be a good source of such
analysis, yet these texts tend to be charged
with multiple emotions, with triggers scatter-
ing across multiple sentences. This paper takes
a novel angle, namely, emotion detection and
trigger summarization, aiming to both detect
perceived emotions in text, and summarize
events and their appraisals that trigger each
emotion. To support this goal, we introduce
COVIDET (Emotions and their Triggers dur-
ing Covid-19), a dataset of ~1,900 English
Reddit posts related to COVID-19, which con-
tains manual annotations of perceived emo-
tions and abstractive summaries of their trig-
gers described in the post. We develop strong
baselines to jointly detect emotions and sum-
marize emotion triggers. Our analyses show
that COVIDET presents new challenges in
emotion-specific summarization, as well as
multi-emotion detection in long social media
posts.
1 Introduction
Large-scale crises such as the COVID-19 pandemic
continuously cause emotional turmoil worldwide.
People are emotionally affected in different ways,
e.g., online education has led to mental health is-
sues among students (Akpınar et al.,2021) as well
as parents (Cui et al.,2021); lock-down policies are
protective for the vulnerable (Flaxman et al.,2020;
Hsiang et al.,2020) while economically disastrous
for many (Odii et al.,2021). Emotion analysis —
both detecting emotion and understanding what
triggers the emotion — brings invaluable insights
both practically (e.g., for first responders, coun-
selors, etc) and in scientific research (Arora et al.,
*Hongli Zhan and Tiberiu Sosea contributed equally.
Reddit Post
1: My sibling is 19 and she constantly goes places with her friends and to there
houses and its honestly stressing me out.
2: Our grandfather lives with us and he has dementia along with other health issues
and my mom has diabetes and heart problems and I have autoimmune diseases
& chronic health issues.
3: She also has asthma.
4: Its stressing me out because despite this she seems to not care about how badly
it would affect all of us if we were to get the virus.
5: And sadly I feel like its not much I can do she literally doesn’t respect my mom
and though I’m older she doesn’t respect me either.
6: Its so frustrating.
Emotions and Abstractive Summaries of Triggers
Emotion: anger
Abstractive Summary of Trigger: My sister having absolutely no regard for any of
our family’s health coupled with the fact that I can’t do anything about it is so
aggravating to me.
Emotion: fear
Abstractive Summary of Trigger: My sibling, who, in spite of our family’s myriad
of issues that all make us high-risk people, continuously goes out and about, which
makes her likely to get infected. I am scared for all of us right now.
Figure 1: An example from COVIDET, with perceived
emotion(s) identified and their trigger(s) summarized.
2021;Uban et al.,2021).
While emotion detection (typically formulated
as a classification task among standard emotion tax-
onomies) is a well-established task (e.g., Mihalcea
and Strapparava (2012); Wang et al. (2012); Abdul-
Mageed and Ungar (2017); Khanpour and Caragea
(2018); Demszky et al. (2020) and in crises con-
texts (Desai et al.,2020;Sosea et al.,2022)), fewer
have studied
what leads to these emotions
in the
scope of the text concerned in a data-driven man-
ner. Xia and Ding (2019) adopt an extraction setup
to identify emotion “causes” that is limited to the
clause level, where only one (explicitly expressed)
emotion and one cause are associated. This does
not generalize to long, spontaneous social media
posts that are emotionally charged. Illustrated in
Figure 1, distinct emotions are triggered by differ-
ent events across multiple sentences.
Additionally, how these events are subjectively
evaluated, interpreted or appraised, e.g., “I can’t
do anything about it” in the first example of Fig-
ure 1, also contribute to the emotion (Smith and
Ellsworth,1985;Ellsworth and Scherer,2003).
The fact that different individuals may have distinct
appraisals towards the same event (Moors et al.,
arXiv:2210.12531v1 [cs.CL] 22 Oct 2022
2013) further highlights the challenging nature of
understanding what triggers an emotion.
In this work we take a novel view, and formulate
emotion-trigger detection as an abstractive summa-
rization task that synthesizes a natural language
description of the events and their appraisals that
trigger a particular emotion. We frame our work
as emotion detection and trigger summarization
(Figure 1), which entails both detecting perceived
emotions in text, and summarizing triggers for each
emotion.
We present COVIDET (
E
motions and their
T
riggers during
Covid
-19), a new dataset sourced
from
1,883
English Reddit posts about the COVID-
19 pandemic. Each post is annotated with 7 fine-
grained emotion labels; for each emotion, annota-
tors provided a concise, abstractive summary de-
scribing the triggers of the emotion. The triggers
are further validated in a separate stage. COVIDET
spans from June 2021 to January 2022, captur-
ing various significant events as well as how they
were emotionally appraised during the pandemic.
Compared to prior emotion studies that consider
only sentence-level texts (Sosea and Caragea,2020;
Demszky et al.,2020) or (short) tweets (Sosea et al.,
2022;Abdul-Mageed and Ungar,2017), COVIDET
is challenging as it contains significantly longer
texts. We showcase examples of COVIDET in Ap-
pendix §A.
Analyses of COVIDET reveal that negative emo-
tions such as fear and anger are prevalent. These
emotions co-occur most frequently with anticipa-
tion, which consistently rise after the Omicron sub-
variant became more dominant with fear dropping.
Topic modeling over the trigger summaries points
to irritations toward those who don’t mask or get
vaccinated, and positivity towards the vaccines.
Using COVIDET, we benchmark models for
emotion detection and emotion-trigger summariza-
tion. We employ both separate emotion detection
and trigger summarization models, as well as joint
models that we designed to simultaneously detect
emotions and generate trigger summaries. Our ex-
periments showcase the distinct nature of our task,
emphasizing that COVIDET is vital to training re-
liable emotion detection and trigger summariza-
tion approaches in a Covid-19 context. COVIDET
bears various unique characteristics, ranging from
its long sequences and invaluable context to the na-
ture of the task itself. Therefore, general emotion
detection or summarization models unsurprisingly
lag behind in performance compared to our meth-
ods. Moreover, human evaluation of the generated
trigger summaries tailored for emotion-trigger sum-
marization indicates that our models are effective
in capturing the underlying triggers of the post.
We release COVIDET and our code at
https:
//github.com/honglizhan/CovidET.
2 Related Work
Summarization.
Recent pre-trained models led
to substantial progress in single document summa-
rization. In the case of abstractive summarization,
encoder-decoder transformer models are used to
synthesize a concise description of the most salient
concepts in the input (Lewis et al.,2020;Zhang
et al.,2020). Significant efforts in summarization
focus on news because of the availability of large
datasets such as CNN/DailyMail (Hermann et al.,
2015) and XSum (Narayan et al.,2018); in the do-
main of social media, TL;DR sentences has been
mined in Reddit to serve as summaries and train
models (Völske et al.,2017;Kim et al.,2019).
However, generic summaries tend not to be infor-
mative if users are concerned with specific emo-
tions expressed.
In this sense our setup fits into settings where
only a certain part of the content is of interest to the
user. We could view our task as answering a query,
“Why does the writer feel [emotion]?”. However,
such queries are more general than query-based
summarization (Daumé III and Marcu,2006;Otter-
bacher et al.,2009;Schilder and Kondadadi,2008;
Nema et al.,2017;Baumel et al.,2018;Laskar
et al.,2020;Su et al.,2021;Zhong et al.,2021),
where queries tend to be more document-specific.
Perhaps a closer task is opinion summarization, or
aspect-based summarization more generally. In
opinion summarization, models need to summarize
affect/opinions about a certain aspect of a service or
product (Popescu and Etzioni,2005;Angelidis and
Lapata,2018;Huy Tien et al.,2019;Suhara et al.,
2020;Angelidis et al.,2021;Amplayo and Lapata,
2021); on the contrary, our setup entails identify-
ing the emotions and summarizing the events and
how they were made sense of with respect to each
emotion. In aspect-based summarization, existing
work has explored summarizing with respect to pre-
designated aspects of certain news (Frermann and
Klementiev,2019;Ahuja et al.,2022), and entities
mentioned in text (Maddela et al.,2022).
Emotion Cause Extraction.
Emotion Cause Ex-
traction (ECE) is a task that aims to extract the
events triggering a particular emotion (Khunteta
and Singh,2021). ECE was first introduced by Lee
et al. (2010), where they defined the task as extract-
ing word-level causes to the given emotion in text.
Chen et al. (2010) and Gui et al. (2016) expanded
the task to clause-level cause detection; Xia and
Ding (2019) aimed to removed the constraint that
emotions must be human-annotated before conduct-
ing automatic cause extraction, and thus proposed
Emotion-Cause Pair Extraction (ECPE) aiming to
extract potential pairs of emotions and causes in
a document. Most of the datasets are in Chinese,
in either micro-blog or news domains (Gao et al.,
2015;Gui et al.,2016;Gao et al.,2017).
In contrast, we study a more generalized no-
tion of triggers of an emotion where readers are
asked to actively appraise and interpret the emo-
tions together with their stimuli in the document,
rather than solely identifying the events behind
each emotion. We use abstractive summarization
to handle triggers, which can better synthesize inter-
connected complex events and abstract concepts,
as well as making the output contextually indepen-
dent.
3 Dataset Construction
We present COVIDET, a novel dataset from En-
glish Reddit posts that is manually annotated with
emotions and summaries of their triggers. This sec-
tion discusses the data creation process; in §4, we
discuss inter-annotator agreement and our human
verification process.
3.1 Selecting & Curating Reddit Posts
We gather posts from
r/COVID19_support1
. We
select it as the source of our data because of its rich
personal narration: rather than COVID-19 news
snippets, this subreddit is targeted for people seek-
ing any community support during the pandemic.
We randomly sample posts before (from Jun 23,
2021 to Oct 1, 2021) and after (from Dec 1, 2021
to Jan 25, 2022) Omicron, a COVID-19 variant that
emerged during December 2021.
We restrict posts to be between
50
-
400
tokens
long (punctuation excluded); this allows us to have
posts that are long enough, but still manageable
for crowdsourcing tasks. Close scrutiny shows that
the posts in COVIDET center around 100 tokens in
1https://www.reddit.com/r/COVID19_support/
50 100 150 200 250 300 350 400
0
50
100
150
200
Count
Figure 2: Distribution of the length of posts in
COVIDET.
length; the distribution of the length of the posts
is given in Figure 2. The average length of posts
in COVIDET is
156.4
tokens (std.dev =
83.3
). We
mask web links with an
[url]
token and do not
provide the metadata to our annotators. Note that 6
posts have length under 50 tokens: this is because
we performed
[url]
masking after length filtering
when collecting the source data. Details of the full
preprocessing procedure are provided in Appendix
§B.
3.2 Annotation Task
Instructions.
Annotators are first asked to anno-
tate Plutchik basic emotions (Plutchik,2001) they
perceive: anger, anticipation, joy, trust, fear, sad-
ness, and disgust.
2
Multiple selection is allowed,
and we also provide a none of the above option in
case no emotion is perceived.
Once the annotators select an emotion, they are
asked to summarize the trigger(s) to their perceived
emotions, specifically an abstractive summary in
their own words, in the author’s voice. The sum-
maries should contain trigger(s) to the emotion
rather than just reflecting the emotion itself. We
provide the detailed instructions to our annotation
task in Appendix §C.
Annotators
We recruit two different groups of
annotators. The first group consists of trained turk-
ers from Amazon Mechanical Turk. The workers’
locale is the US, and they have completed
500
+
HITs with an acceptance rate
≥95
%. This group
contributes to COVIDET’s training and validation
sets. The second group consists of 2 linguistic un-
dergraduate students, who contributes to the test set.
To ensure the quality of COVIDET, both groups
2
After annotation, we found very little surprise in the data
(59 in total), thus we leave out surprise for this work.
of annotators are trained and qualified in a pre-
qualification process. We also ask them to revise
their work when needed during annotation.
Pre-Annotation Training
We trained the anno-
tators before they annotate COVIDET. We set up a
qualification task on the Amazon Mechanical Turk.
The qualification task involves 3 posts, and annota-
tors are required to complete the qualification task.
Through manually examining the annotators’ work
on the qualification task and comparing the annota-
tions to the gold annotations we develop, we filter
high-quality annotators and give them the access to
our annotation task. We also provide feedback to
their annotations. The turkers are paid at least $10
per hour. To ensure this goal is reached, we keep
track of their working time on the backstage and
give out bonuses accordingly when needed.
Annotation Revisions
During the process of the
annotation on COVIDET, we regularly review the
annotations and give feedback accordingly. When
needed, we send the annotations back to the anno-
tator along with the original post, and ask them to
revise their work based on our suggestions. Note
that the annotator is responsible for the revision of
their own work only.
3.3 Benchmark Dataset
We annotated
1,485
posts on the Amazon Mechan-
ical Turk, each annotated by two independent work-
ers. Since the neutral class is very infrequent, we
remove it from our experiments. To facilitate our
experiments, we split the examples into
1,200
ex-
amples for training and
285
examples for valida-
tion. Our test set—which is annotated by linguistic
undergraduates—contains 398 examples.
If at least one annotator labels a post with an
emotion
e
, then we include emotion
e
as an emo-
tion label. In cases where both annotators assign
an emotion
e
to a post, we consider the trigger sum-
maries as two separate training examples for the
trigger summarization task. In cases where a post
has two different trigger summaries in the valida-
tion or the test set, we consider them as multiple
references when computing our metrics.
4 Agreement and Validation
To account for the quality of COVIDET, we mea-
sure the inter-annotator agreement in emotions
(§4.1) and triggers (§4.2). The annotations are
further validated through human inspection in §4.3.
0 200 400 600 800 1000 1200
Number of Examples
none
trust
joy
disgust
sadness
anger
fear
anticipation
0.4
0.5
0.6
0.7
0.8
Inter-annotator PEA Score
Figure 3: Emotion distribution of COVIDET, ranked
by the number of examples. Colors indicate the inter-
annotator agreement measured by the PEA score.
Results reveal that annotators tend to agree with
each other in emotions whilst using varied vocabu-
laries when summarizing the triggers.
4.1 Agreement in Emotions
Percentage Overlap.
For each example in
COVIDET, we measure the number of emotions in
which both annotators agree upon. Results show
that in 81.4% of the examples in COVIDET, both
annotators agree on at least 1 emotion label; in
26.6% of the examples, both annotators agree on
at least 2 emotion labels.
PEA Score.
To account for distances between
emotions (e.g., disgust is further away from joy
than from anger), we report the Plutchik Emotion
Agreement (PEA) metric (Desai et al.,2020) for
the inter-annotator agreement of emotions anno-
tated in COVIDET. We first report the average PEA
score among annotators weighted by their numbers
of annotations, which is 0.8 for the training and
validation sets combined, and 0.821 for the test
set (0.804 for all three combined). These numbers
indicate high agreement (Desai et al.,2020).
Figure 3shows per-emotion PEA scores, along
with the frequency of each emotion. All emotions
have high agreement; the highest are among fear
and anger, with the average PEA scores at around
0.85; the lowest is trust, with the average PEA
score at around 0.74.
Finally, to calculate agreement between students
and crowd workers, we randomly sample 208 exam-
ples from the training set and ask the linguistic un-
dergraduate students to annotate them from scratch.
We assign one student per example for validation.
The average PEA score between crowd workers
and linguistics students is 0.832, suggesting high
摘要：

WhyDoYouFeelThisWay?SummarizingTriggersofEmotionsinSocialMediaPostsHongliZhan1TiberiuSosea2CorneliaCaragea2JunyiJessyLi11DepartmentofLinguistics,TheUniversityofTexasatAustin2DepartmentofComputerScience,UniversityofIllinoisChicago{honglizhan,jessy}@utexas.edu{tsosea2,cornelia}@uic.eduAbstractCrises...

展开>> 收起<<
Why Do You Feel This Way Summarizing Triggers of Emotions in Social Media Posts Hongli Zhan1Tiberiu Sosea2Cornelia Caragea2Junyi Jessy Li1.pdf

共19页,预览4页

还剩页未读， 继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

相关推荐

更多
立即下载
分类：图书资源 价格：10玖币 属性：19 页 大小：2.86MB 格式：PDF 时间：2025-05-06

开通VIP享超值会员特权

  • 多端同步记录
  • 高速下载文档
  • 免费文档工具
  • 分享文档赚钱
  • 每日登录抽奖
  • 优质衍生服务

作者详情

相关内容

更多

热门标签

人际关系 配电装置 动力学连接体 力的合成 高考理综 全宋诗作者索引 公务员考试
/ 19
评分 收藏
分享
立即下载
客服
关注