Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios Zhuohao Chen1 Nikolaos Flemotomos1 Zac E. Imel2 David C. Atkins3

2025-04-29 2 0 323.25KB 9 页 10玖币

侵权投诉

Leveraging Open Data and Task Augmentation to Automated Behavioral

Coding of Psychotherapy Conversations in Low-Resource Scenarios

Zhuohao Chen1, Nikolaos Flemotomos1∗

, Zac E. Imel2, David C. Atkins3,

Shrikanth Narayanan1

1University of Southern California, Los Angeles, CA, USA

2University of Utah, Salt Lake City, UT, USA

3University of Washington, Seattle, WA, USA

sail.usc.edu zac.imel@utah.edu datkins@u.washington.edu

Abstract

In psychotherapy interactions, the quality of

a session is assessed by codifying the com-

municative behaviors of participants during

the conversation through manual observation

and annotation. Developing computational ap-

proaches for automated behavioral coding can

reduce the burden on human coders and facil-

itate the objective evaluation of the interven-

tion. In the real world, however, implementing

such algorithms is associated with data spar-

sity challenges since privacy concerns lead to

limited available in-domain data. In this paper,

we leverage a publicly available conversation-

based dataset and transfer knowledge to the

low-resource behavioral coding task by per-

forming an intermediate language model train-

ing via meta-learning. We introduce a task aug-

mentation method to produce a large number

of “analogy tasks” — tasks similar to the tar-

get one — and demonstrate that the proposed

framework predicts target behaviors more ac-

curately than all the other baseline models.

1 Introduction

Advances in spoken language processing tech-

niques have improved the quality of life across

several domains. One of the striking applications is

automated behavioral coding in the ﬁelds of health-

care conversations such as psychotherapy. Behav-

ioral coding is a procedure during which experts

manually identify and annotate the participants’

behaviors (Cooper et al.,2012). However, this pro-

cess suffers from a high cost in terms of both time

and human resources (Fairburn and Cooper,2011).

Building computational models for automated be-

havioral coding can signiﬁcantly reduce the cost in

time and provide scalable analytical insights into

the interaction. A great amount of such work has

been developed, including for addiction counseling

∗

Work done while Nikolaos Flemotomos was at Univer-

sity of Southern California in 2022, and he is now afﬁliated to

Apple Inc.

(Tanana et al.,2016;Pérez-Rosas et al.,2017;Chen

et al.,2019;Flemotomos et al.,2022) and couples

therapy (Li et al.,2016;Tseng et al.,2016;Big-

giogera et al.,2021). However, automated coding

is associated with data sparsity due to the highly

sensitive nature of the data and the costs of human

annotation. Due to those reasons, both samples and

labels of in-domain data are typically limited. This

paper aims to train computational models for pre-

dicting behavior codes directly from psychotherapy

utterances through classiﬁcation tasks with limited

in-domain data.

Recently, substantial work has shown the suc-

cess of universal language representation via pre-

training context-rich language models on large cor-

pora (Peters et al.,2018;Howard and Ruder,2018).

Particularly, BERT (Bidirectional Encoder Repre-

sentations from Transformers) has achieved state-

of-the-art performance in many natural language

processing (NLP) tasks and provided strong base-

lines in low-resource scenarios (Devlin et al.,2019).

However, these models rely on self-supervised pre-

training on a large out-of-domain text corpus. In

prior works, the data sparsity issue has also been

addressed by introducing an intermediate task pre-

training using some other high-resource dataset

(Houlsby et al.,2019;Liu et al.,2019;Vu et al.,

2020). However, not all the source tasks yield posi-

tive gains. Sometimes the intermediate task might

even lead to degradation due to the negative trans-

fer (Pruksachatkun et al.,2020;Lange et al.,2021;

Poth et al.,2021). To improve the chance of ﬁnding

a good transfer source, we need to collect as many

source tasks as possible. Another approach is meta-

learning which aims to ﬁnd optimal initialization

for ﬁne-tuning with limited target data (Gu et al.,

2018;Dou et al.,2019;Qian and Yu,2019). This

approach also calls for enough source tasks and is

affected by any potential task dissimilarity (Jose

and Simeone,2021;Zhou et al.,2021).

The challenge we need to handle is that both

arXiv:2210.14254v1 [cs.CL] 25 Oct 2022

Code Description #Train #Test

Therapist Utterances

FA Facilitate 19397 5838

GI Giving information 17746 5064

RES Simple reﬂection 7236 2137

REC Complex reﬂection 4974 1510

QUC Closed question 6421 1569

QUO Open question 5011 1475

MIA MI adherent 4898 1346

MIN MI non-adherent 1358 237

Patient Utterances

FN Follow/Neutral 56204 15426

POS Change talk 6146 1737

NEG Sustain talk 5121 1407

Table 1: Data statistics for behavior codes in Motiva-

tional Interviewing psychotherapy.

utterances and assigned codes in psychotherapy in-

teractions are domain-speciﬁc, making it difﬁcult to

leverage any open resource from a related domain.

Considering that psychotherapy counseling takes

place in a conversational setting, here we use a

publicly available dataset — Switchboard-DAMSL

(SwDA) corpus (Stolcke et al.,2000) — for the

intermediate stages of knowledge transfer. Unlike

most previous meta-learning frameworks, which

require auxiliary tasks from various datasets, our

work uses only one dataset and produces the source

tasks by a task augmentation procedure. The task

augmentation framework evaluates the correlations

between the source and target labels. It produces

source tasks by choosing subsets of source labels

whose classes are in one-to-one correspondence

with the target classes. Using this strategy, we can

generate a large number of source tasks similar to

the target task and thus improve the performance

of meta-learning. The experimental results show

that incorporating our proposed task augmentation

strategy into meta-learning enhances the classiﬁca-

tion accuracy of automated behavioral coding tasks

and outperforms all the other baseline approaches.

2 Dataset

We use data from Motivational Interviewing (MI)

sessions of alcohol and drug abuse problems (Baer

et al.,2009;Atkins et al.,2014) for the target task.

The corpus consists of 345 transcribed sessions

with behavioral codes annotated at the utterance

level according to the Motivational Interviewing

Skill Code (MISC) manual (Houck et al.,2010).

We split the data into training and testing sets with

a roughly 80%:20% ratio across speakers, resulting

in 276 training sessions and 67 testing sessions.

The statistics of the data are shown in Table 1.

We perform the intermediate task with the SwDA

dataset, which consists of telephone conversations

with a dialogue act tag for each utterance. We

concatenate the parts of an interrupted utterance

together, following Webb et al. (2005), which re-

sults in 196K training utterances and 4K testing

utterances. This dataset supports 42 distinct tags,

with more details displayed in Appendix A.

3 Methodology

3.1 Task Augmentation via Label Clustering

We deﬁne a low resource target task on

X × Y

and use

x∈ X

to denote data and

y∈ Y ={1,2, ..., M}

to denote the target labels.

We additionally assume a data-rich source task de-

ﬁned on

X × Z

with samples

{(x1, z1),(x1, z2),

..., (xn, zn)}

supported by a much larger label set

denoted by

z∈ Z ={1,2, ..., N}

N > M

. Our

task augmentation procedure aims at producing nu-

merous tasks similar to the target task—we will

refer to those as the “analogy tasks”.

The high-level idea is to construct the tasks

with class labels similar to the target ones. Thus

we explore the relationships between

and

We initialize

label subsets

C1=∅, C2=

∅,· · · , CM=∅

to gather the source labels

corresponding to

y= 1, y = 2,· · · , y =M

respectively. In the ﬁrst step, we ﬁne-tune on

the in-domain target data to achieve a dummy

classiﬁer

. Then, we feed the source sam-

ples into

and obtain the predicted labels

{f(x1), f(x2),· · · , f(xn)}

. For any pair of a tar-

Algorithm 1 Construction of Analogy Tasks

Initialize model parameters θ;K, M ∈N.

Create empty label subsets:

C1=∅, C2=

∅, ..., CM=∅.

Fine-tune BERT with in-domain samples to ob-

tain the classiﬁer f

for i= 1 to Kdo

for j= 1 to Mdo

For the target label

y=j

, select

z∗∈ Z

by Equation(1) and (2), then add it to Cj

Remove the label z∗from Z

Select one label from

C1, C2, ..., CM

to produce

MKanalogy tasks

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LeveragingOpenDataandTaskAugmentationtoAutomatedBehavioralCodingofPsychotherapyConversationsinLow-ResourceScenariosZhuohaoChen1,NikolaosFlemotomos1,ZacE.Imel2,DavidC.Atkins3,ShrikanthNarayanan11UniversityofSouthernCalifornia,LosAngeles,CA,USA2UniversityofUtah,SaltLakeCity,UT,USA3UniversityofWashing...

展开>> 收起<<

Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios Zhuohao Chen1 Nikolaos Flemotomos1 Zac E. Imel2 David C. Atkins3.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios Zhuohao Chen1 Nikolaos Flemotomos1 Zac E. Imel2 David C. Atkins3

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: