DEPTWEET A Typology for Social Media Texts to Detect Depression Severities_2

2025-05-06 0 0 1.65MB 17 页 10玖币

DEPTWEET: A Typology for Social Media Texts to Detect Depression

Severities

Mohsinul Kabira,Tasnim Ahmeda,Md. Bakhtiar Hasana,Md Tahmid Rahman Laskarb,Tarun

Kumar Joarderc,Hasan Mahmudaand Kamrul Hasana

aDepartment of Computer Science and Engineering, Islamic University of Technology, Board Bazar, Gazipur, 1704, Dhaka, Bangladesh

bDepartment of Computer Science, York University, 4700 Keele St, Toronto, M3J 1P3, Ontario, Canada

cDepartment of Psychology, University of Rajshahi, Matihar, Rajshahi, 6205, Bangladesh

ARTICLE INFO

Keywords:

Social Media

Mental Health

Depression Severity

Dataset

ABSTRACT

Mental health research through data-driven methods has been hindered by a lack of standard typology

and scarcity of adequate data. In this study, we leverage the clinical articulation of depression to build a

typology for social media texts for detecting the severity of depression. It emulates the standard clinical

assessment procedure Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and Patient

Health Questionnaire (PHQ-9) to encompass subtle indications of depressive disorders from tweets.

Along with the typology, we present a new dataset of 40191 tweets labeled by expert annotators. Each

tweet is labeled as ‘non-depressed’ or ‘depressed’. Moreover, three severity levels are considered

for ‘depressed’ tweets: (1) mild, (2) moderate, and (3) severe. An associated conﬁdence score is

provided with each label to validate the quality of annotation. We examine the quality of the dataset

via representing summary statistics while setting strong baseline results using attention-based models

like BERT and DistilBERT. Finally, we extensively address the limitations of the study to provide

directions for further research.

1. Introduction

Analyzing the presence of mood and psychological dis-

orders through behavioral and linguistic cues from social

media data remains a critical area of interdisciplinary re-

search. In addition to these disorders, the last decade has seen

exponentially increasing attempts to assess related symp-

tomatology such as depressive disorders, self-harm, and

severity of mental illness using non-clinical data (Bucci

et al.,2019). Social media platforms and other online discus-

sion forums have been particularly appealing to the research

community for various research purposes (e.g., population-

level mental health monitoring (Conway and O’Connor,

2016), personal traits detection (Marouf et al.,2020), cy-

berbullying spotting (Bozyiğit et al.,2021), etc.) because

of the massive scale of data. This massive data ﬂow has

resulted from increasing rates of internet access and people

spontaneously sharing their suﬀering, pain, and struggle

anonymously on these platforms (Ofek et al.,2015). Rec-

ognizing the early symptoms of depressive disorder through

a person’s language use can prevent many disastrous out-

comes like self-harm, suicide, etc., and even help deploy

eﬀective treatment in proper time. Moreover, the outbreak

of the COVID-19 pandemic is likely to have devastating

impacts on the mental health of millions of individuals as

lockdown in the aﬀected areas has reported in high rises in

the incident rates of mood disorder, including acute stress

disorder, post-traumatic stress disorder, generalized anxiety

mohsinulkabir@iut-dhaka.edu (M. Kabir);

tasnimahmed@iut-dhaka.edu (T. Ahmed); bakhtiarhasan@iut-dhaka.edu

(Md.B. Hasan); tahmedge@cse.yorku.ca (M.T.R. Laskar);

tarun_psy@ru.ac.bd (T.K. Joarder); hasan@iut-dhaka.edu (H. Mahmud);

hasank@iut-dhaka.edu (K. Hasan)

ORCID(s):

disorder, and overall sub-clinical mental health deterioration

(Singh et al.,2020). The scope of mental health deterioration

during the COVID-19 pandemic and the comprehensive

nature of diagnosing depressive disorders have provided an

unprecedented need to infer the mental states of individuals

from all-inclusive resources. Recent studies have revealed

that valuable insights into the impact of the pandemic on

population-level mental health can be inferred from posts or

comments on social media (Low et al.,2020).

A persistent challenge for the researchers speciﬁc to the

mental health space is the need to: (a) establish a typology

for text contents on social media to detect the severity of

mental illness with clinical validation and robustness (Ernala

et al.,2019), and (b) reliably apply this typology to obtain a

suﬃcient sample size of high-quality data. Prior research has

explored opportunities to capture mental health states from

social media data using regular expressions to identify self-

reported diagnosis or by using vectorization-driven methods

to cluster activity patterns of users. However, deliberately

relying on self-labeled data or unsupervised clustering leads

to oversimpliﬁcation and lacks clinical eﬃcacy (Ernala et al.,

2019). Practical exertion of mental health research includes

identifying risky behaviors and providing timely interven-

tions such as suicide prevention eﬀorts adopted by Facebook

(Vincent,2017). The availability of high-quality, large-scale,

annotated datasets addressing the severity of mental illness

is one of the key elements for advancement on this front.

Unfortunately, there are very few available datasets for de-

pression severity which also lacks strong ground truths based

on clinical validation (Tolentino and Schmidt,2018).

This study aims to contribute in this domain through

(a) establishing a typology for social media contents (i.e.,

tweet text) built upon a psychological theory for detecting

Kabir et al.: Preprint submitted to Elsevier Page 1 of 17

arXiv:2210.05372v1 [cs.CL] 10 Oct 2022

DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities

the severity of the mental condition of depressed individuals,

(b) constructing a dataset named DEPTWEET1containing

around 40191 tweets with corresponding crowdsourced la-

bels and conﬁdence scores. The labeling typology of the

dataset assigns a higher-level classiﬁcation to each tweet,

such as (1) Non-depressed, (2) Mildly Depressed, (3) Mod-

erately Depressed, and (4) Severely Depressed. There is also

an associated conﬁdence score (between 0.5 and 1) for each

label.

The procedure used to assess the severity of depres-

sion in this study was based on a well-established clinical

assessment method known as the Diagnostic and Statisti-

cal Manual of Mental Disorders, Fifth Edition (DSM-5)

(Arbanas,2015), and it was carried out under the supervi-

sion of two expert clinical psychologists. The DEPTWEET

dataset contributes further high-quality data on attributes

like none, mild, moderate or severe depression, adding to

existing datasets on these and related attributes (Ahmed

et al.,2021b;Mukhiya et al.,2020), and provides the ﬁrst

dataset of this scale on depression severities to the best of

our knowledge. The approach utilized in this study can be

adopted to generate high-quality mental health data from

various platforms in future investigations. Moreover, given

that the data was collected in the latter half of 2021, topic

modeling on this dataset can provide useful insight into the

impact of the COVID-19 pandemic on individuals’ mental

health.

The remaining sections of the paper are structured as

follows: Section 2and 3outlines the motivation and back-

ground of the DEPTWEET dataset. The data collection,

quality control mechanisms, and the summary statistics of

the data are described in Section 4. The baseline classiﬁca-

tion model for this dataset and evaluation metrics are pre-

sented in Section 5. Section 6discusses the classiﬁcation re-

sults, potential sources of bias in the data, and the necessary

aspects to consider while conducting additional research in

this domain. Finally, Section 7draws a conclusion to the

current study and discusses future directions.

2. Related Work

Computational linguistics techniques are very diﬃcult

to be opted as a complete substitute for in-person mental

illness diagnosis, but the successful application of this do-

main in identifying the progress and level of depression of

individuals in online therapy may provide clinicians with

more insights, allowing them to apply interventions more

eﬀectively and eﬃciently. Studies analyzing web data, espe-

cially social media platforms, have piqued the interest of the

research community due to their scope and deep entangle-

ment in contemporary culture (Fuchs,2015). Coppersmith

et al. (2014) made a prominent contribution in this domain

by developing a procedure of extracting mental health data

from social media. In their study, tweets were crawled from

1The DEPTWEET dataset is available at

https://github.com/mohsinulkabir14/DEPTWEET

user proﬁles who publicly stated that they had been diag-

nosed with various mental illnesses on their Twitter feed.

They mixed control samples from the general population

(people who are not depressed) with the tweets of the self-

reported diagnosed group. Additionally, they conducted an

LIWC (Linguistic Inquiry Word Count) analysis to measure

deviations of each disorder group from the control group.

They focused on the analysis of four mental illnesses: Post-

Traumatic Stress Disorder (PTSD), Depression, Bipolar Dis-

order, and Seasonal Aﬀective Disorder (SAD), and proposed

this novel method to gather data for a range of mental

illnesses quickly and cheaply. Numerous studies later fol-

lowed this approach to detect relevant mental health data for

various mental illnesses. For example, The Computational

Linguistics and Clinical Psychology (CLPsych) 2015 shared

task (Coppersmith et al.,2015) collected self-reported data

on Depression and PTSD. They further annotated the data

with human annotators to remove jokes, quotes, etc., from

the collected data. The shared task participants had three

binary classiﬁcation tasks- identify depression vs. control,

identify PTSD vs. control, and identify depression vs. PTSD.

These datasets were used in a variety of studies to discover

patterns in the language use of users suﬀering from various

mental illnesses (Pedersen,2015;Coppersmith et al.,2016;

Amir et al.,2017). In particular, Resnik et al. (2015) con-

ducted several topic modeling (supervised Latent Dirichlet

Allocation (LDA), supervised anchor topic modeling, etc.)

to diﬀerentiate the language usage of depressed and non-

depressed individuals using the datasets of Coppersmith

et al. (2014) and CLPSych Shared Task (2015).

Following a similar approach, Chen et al. (2018) col-

lected tweets from self-reported depressed users and investi-

gated the potential of non-temporal and temporal measures

of emotions over time to identify depression symptoms from

their tweets by detecting eight basic emotions (e.g. anger,

fear, etc.). Additionally, classiﬁers were built to label Twitter

users as either depressed or non-depressed (control) groups

calculating the strength scores based on the intensity of each

emotion and a time series analysis of each user. Among other

social medias, Tian et al. (2016) explored sleep complaints

on Sina Weibo (a Chinese microblogging website) to dis-

cover users’ diurnal activity patterns and gain insight into the

mental health of insomniacs. Twitter data on mental health

had also been collected, with speciﬁc Twitter campaigns

being targeted. For instance, Jamil et al. (2017) prepared a

dataset from the users who participated in the #BellLetsTalk

2015 campaign that was inaugurated to promote awareness

about mental health issues. They collected public tweets

from 25362 Canadian users and built a user-level classiﬁer

to detect at-risk users and a tweet-level classiﬁer to predict

symptoms of depression in tweets. From this campaign, they

came across only 5% tweets that talk about depression and

95% non-depressed tweets. While these methods can extract

large volumes of data for a low cost, they do not ensure a

suﬃcient sample of interest and have inevitably resulted in a

low number of positive samples (mental-health related data).

Kabir et al.: Preprint submitted to Elsevier Page 2 of 17

DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities

Several previous studies have investigated the use of

clinical methodologies along with data mining tools to ex-

tract depression symptoms from diverse sources. Yazdavar

et al. (2017) created a lexicon of depression symptoms based

on the nine disorders described in the clinically established

Patient Health Questionnaire (PHQ-9) and utilized this to

ﬁnd symptoms of depression in tweets from users with self-

reported depressive symptoms in their Twitter proﬁle. They

also developed a statistical model to categorize and monitor

depressive symptoms for continuous temporal analysis of an

individual’s tweets. In a similar study, Mukhiya et al. (2020)

proposed an open set of depression word embeddings that

extracts depression symptoms from patient-authored text

data based on PHQ-9 to deliver personalized intervention

to people with symptoms of depression. Yadav et al. (2020)

utilized the nine symptom classes of the PHQ-9 question-

naire to manually annotate the tweets collected from 205

self-reported depression diagnosed users. Their proposed

framework took into consideration the ﬁgurative language

(metaphor, sarcasm etc) wired in the communication of de-

pressive users on Twitter. Ahmed et al. (2021b) extracted de-

pression symptoms in patient authored text in a similar fash-

ion with PHQ-9 questionnaire but used an attention-based

in-depth entropy active learning to annotate the unlabeled

texts automatically. Their mechanism increased the trainable

instances of mental health data using a semantic clustering

mechanism with to reduce the data annotation task. Another

mental health tool used by psychiatrists, namely the Diag-

nostic and Statistical Manual of Mental Disorders (DSM-5),

has also been used to categorize mental disorders from social

media content. Gaur et al. (2018) developed an approach

to map subreddits into DSM-5 categories. They created a

lexicon from various subreddit posts by extracting n-grams

and topics using LDA and mapped this lexicon with DSM-5

lexicon created by available medical knowledge bases (ICD-

102, SNOMED-CT3, DataMed4). Their approach attempted

to connect a patient on social media platforms such as Reddit

to appropriate mental health resources and to provide web-

based intervention. Cavazos-Rehg et al. (2016) investigated

the most common themes of depression-related chatter on

Twitter that corresponded to the DSM-5 symptoms for major

depressive disorder. While these methods may have clinical

validity, most studies that use them lack suﬃcient ground

truth data due to the absence of a thorough annotation

procedure.

Very few studies have investigated predicting the sever-

ity of depression based on users’ language usage on web plat-

forms. De Choudhury et al. (2013) proposed a metric named

social media depression index (SMDI) using a probabilistic

model to help characterize the levels of depression in the

population level. This probabilistic model is an SVM clas-

siﬁer that can predict whether or not a Twitter post contains

symptoms of depression. To construct and train this model,

2https://bioportal.bioontology.org/ontologies/ICD10

3http://bioportal.bioontology.org/ontologies/SNOMEDCT

4https://datamed.org/

they collected data using crowdsourcing technique and de-

rived various linguistic and network features (e.g., number of

followers) from tweets of individuals suﬀering from clinical

depression, which was measured using the CES-D (Center

for Epidemiologic Studies Depression Scale) screening test

(Radloﬀ,1977). Schwartz et al. (2014) attempted to predict

and characterize the severity of depression based on people’s

Facebook language use. They gathered survey responses and

Facebook posts from 28749 Facebook users and trained a

classiﬁcation model to predict depression symptoms using

n-grams, linguistic behavior, and LDA topics. They tried to

quantify the seasonal changes in depression symptoms based

on social media posts and discovered that symptoms increase

from summer to winter. These approaches had the potential

to generate a large dataset with good quality data if they

were developed in partnership with expert psychologists and

domain experts.

While previous research has made signiﬁcant progress

toward automatic depression assessment tools based on so-

cial media, some limitations have been identiﬁed through

critical evaluation. Most previous works have relied on self-

reported depressed user proﬁles when it comes to data

extraction. While this is an inexpensive way to gather a

massive scale of data, it doesn’t guarantee enough sam-

ples with depressive symptoms without manual intervention.

Also, this approach might lack enough clinical validation to

extract depression symptoms. Studies that leveraged clin-

ical assessment tools to extract data, such as the PHQ-9

or DSM-5, lacked supervision from domain experts and

mostly annotated their data in an automated manner, such as

using unsupervised topic modeling or clustering techniques.

Moreover, only a few studies have investigated how to collect

data on diﬀerent depression severities with suﬃcient clinical

eﬃcacy. The existing datasets only concentrate on binary

detection of whether a particular tweet manifests depres-

sion or not, the severity level of which is mostly ignored.

This might lead to models competent enough in detecting

subtle cues of depression turn a blind eye towards them. A

dataset containing suﬃcient samples to train large models

with strong ground truth labels depicting the severity of

depression can go a long way to alleviate these issues.

3. Measuring Severity of Depression

In the current study, a user posting a tweet on social

networking site Twitter is considered to be depressed if the

tweet depicts behaviors portraying symptoms of depression.

Such a tweet may not necessarily be complete, contain well-

structured sentences, or even grammatically correct, making

the task even more diﬃcult.

According to the Diagnostic and Statistical Manual of

Mental Disorders (DSM), clinical depression can be diag-

nosed considering the existence of a set of symptoms over

a substantial amount of time (Yazdavar et al.,2017). Incor-

porating this idea, the Patient Health Questionnaire (PHQ-

9) (Kroenke et al.,2001) provides a set of questionnaires,

which is widely used to screen, diagnose and measure the

Kabir et al.: Preprint submitted to Elsevier Page 3 of 17

DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities

Table 1

Sample tweets, seed terms and ﬁnal keywords list for each symptom of PHQ-9 Questionnaire

PHQ-9 Symptoms Sample Tweet Seed Terms Final Keyword List

Lack of interest (S1) Am I depressed or am I just

bored? Apathy and irony,

postmodern anxiety

disinterest involved, occupied,

pessimism, reversion,

absorbed, lifelessness, bored,

enthusiasm, engrossed,

worried, apathy.

Feeling Down (S2) High functioning

depression, I can’t fester in

my misery but i’m fuckin

miserable

hopeless, depressed dejected, dismayed,

dispirited, demoralized,

grimmed, misery, grim,

downhearted, low-spirited,

bleak, desperate, lost,

frustrated.

Sleep Disorder (S3) forcing myself up now so

I’m not awake when the

power goes oﬀ much later,

lol

awake, sleep nap, restless, awake, whole

night, bedtime.

Lack of Energy (S4) I’m so exhausted and I still

have work 9-5 and then red

rocks day three

tired, energy weary, fatigue, fag, fag out,

overtire, overfatigued,

burned-out, burnt-out,

exhausted, dog-tired,

washed-out, drained,

whacked.

Eating Disorder (S5) another saturday night

where i’m too depressed to

sleep after overeating....i

am extremely bored of this

life

appetite, overeating aversion, distaste, loathing,

malformed, bulimic, puﬀy,

starve, fat

Low Self-estemm (S6) I got on the scale today and

I am disgusted. Like utterly

disgusted. Depression really

beat my ass and had me

slacking

loser, failure loser, relapse, downfall,

ruined, ﬂop, dead-duck,

disappointment, achiever,

misﬁre, underdog,

falling-apart, disgusted

Concentration

Problems (S7)

Whenever it gets close to

my bday I always go

through some type of

cleansing/depression..

Scattered focus...

concentrate, focus immersed, decentralize,

deconcentrate, scattered,

dispersed, unsettled, focus

Hyper/Lower Activity

(S8)

I spend hours of my day

staring at screens,

immobile. Why am I

depressed???

moving, immobile,

restless

discontent, ungratiﬁed,

unsatisﬁed, stand-still,

refrained, immobile

Suicidal Thoughts

(S9)

I know that I can’t undo

The self-destruction, the

damage I’ve done

dead, hurt, suicide trauma, harm, suﬀering,

anguish, hemorrhage,

penetrating-trauma, torment,

agony, excruciate, damaged,

gag, suﬀocate,

self-destruction

severity of depression. Using this set of questionnaires, nine

distinct symptoms related to diﬀerent disorders, such as lack

of interest, eating disorder, etc., can be extracted (Table 1).

The frequency of these symptoms can help classify the

severity of depression as none, mild, moderate, and severe

conditions. This approach is called Clinical Symptom Elici-

tation Process (CSEP) (World Health Organization,1993).

In the current study, this was further extended using the

mood scale provided by BipolarUK5to identify the char-

acteristics related to diﬀerent levels of depression. The fol-

lowing characteristics were then veriﬁed by the collaborator

psychologists and used to detect the level of depression from

the user tweets:

5https://www.bipolaruk.org/faqs/mood-scale

Kabir et al.: Preprint submitted to Elsevier Page 4 of 17

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DEPTWEET:ATypologyforSocialMediaTextstoDetectDepressionSeveritiesMohsinulKabira,TasnimAhmeda,Md.BakhtiarHasana,MdTahmidRahmanLaskarb,TarunKumarJoarderc,HasanMahmudaandKamrulHasanaaDepartmentofComputerScienceandEngineering,IslamicUniversityofTechnology,BoardBazar,Gazipur,1704,Dhaka,BangladeshbDepartm...

展开>> 收起<<

DEPTWEET A Typology for Social Media Texts to Detect Depression Severities_2.pdf

共17页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

DEPTWEET A Typology for Social Media Texts to Detect Depression Severities_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: