Augmenting Task-Oriented Dialogue Systems with Relation Extraction Andrew Lee1Zhenguo Chen2Kevin Leach3Jonathan Kummerfeld1 1University of Michigan

2025-05-02 0 0 306.89KB 9 页 10玖币

侵权投诉

Augmenting Task-Oriented Dialogue Systems with Relation Extraction

Andrew Lee,1Zhenguo Chen,2Kevin Leach,3Jonathan Kummerfeld 1

1University of Michigan

2Bloomberg

3Vanderbilt University

ajyl@umich.edu, Zhenguo.Chen@colorado.edu, kevin.leach@vanderbilt.edu, jkummerf@umich.edu

Abstract

The standard task-oriented dialogue pipeline uses intent clas-

siﬁcation and slot-ﬁlling to interpret user utterances. While

this approach can handle a wide range of queries, it does

not extract the information needed to handle more complex

queries that contain relationships between slots. We propose

integration of relation extraction into this pipeline as an effec-

tive way to expand the capabilities of dialogue systems. We

evaluate our approach by using an internal dataset with slot

and relation annotations spanning three domains. Finally, we

show how slot-ﬁlling annotation schemes can be simpliﬁed

once the expressive power of relation annotations is available,

reducing the number of slots while still capturing the user’s

intended meaning.

1 Introduction

Dialogue platforms like Watson, Rasa, and Dialogﬂow have

enabled a dramatic increase in the development and use of

dialogue systems by bringing together NLP models, data

collection and curation, and scalable deployment (Meteer

et al. 2019). These platforms all follow the same general

framework for interpreting queries, shown in Figure 1, con-

sisting of (1) classiﬁcation models for domain or intent iden-

tiﬁcation and (2) slot-ﬁlling or entity recognition models to

identify relevant entities in a query. While these two models

can handle a wide range of queries, they are unable to handle

more complex queries (Aghajanyan et al. 2020; Davidson,

Yu, and Yu 2019; Gupta et al. 2018). Speciﬁcally, when a

query contains multiple slots with semantic relations, such

as those shown in Table 1, intent classiﬁcation and slot-

ﬁlling cannot easily capture the necessary information.

Meanwhile, Relation Extraction (RE) has been effectively

applied to a range of data sources, such as news articles,

encyclopedia entries, and blog posts, and even conversa-

tional data (Yu et al. 2020). Recently, conversational seman-

tic parsing (Aghajanyan et al. 2020; Cheng et al. 2020; An-

dreas et al. 2020) has also built large end-to-end models to

extract semantic relations. While these representations are

expressive, they have not been integrated into dialogue sys-

tem platforms because of the difﬁculty of collecting new

domain-speciﬁc data.

NLU Module

ASR Domain/

Intent

Identification

Slot Filling Relation

Extraction

Dialogue

Management

Backend

Application

Natural

Language

Generation

TTS

User

Figure 1: The standard architecture of deployed dialogue

systems. We show how adding relation extraction can enable

support for more complex queries.

In this paper, we show how relation extraction can be

incorporated into task-oriented dialogue systems. RE is

a lightweight but powerful way to extend the capabili-

ties of a dialogue system. In particular, suitable data can

be rapidly annotated without extensive training, making

domain-speciﬁc development feasible.

To evaluate our idea, we use an internal conversational

dataset spanning three domains containing multiple slots

and relations. We investigate the accuracy, scalability, and

generalizability of a relation extraction model within a di-

alogue system. Finally we demonstrate the beneﬁts of in-

cluding an RE model in terms of the dialogue system’s

expressive power, meaning the system’s ability to derive

new semantic information that was not seen during train-

ing. Adding relation extraction to the task-oriented dialogue

pipeline will broaden the range of queries these systems can

handle and improve their robustness to variation in the ways

people express relations in their queries.

2 Related Work

We give a brief overview in various ways in which semantic

relations can be extracted in a dialogue setting.

2.1 Relation Extraction

Relation Extraction (RE) models predict labeled links be-

tween spans of text. They have been incorporated into

a range of downstream applications, including question-

answering, ontology population, and knowledge base con-

struction. These have involved applying RE at a range of

arXiv:2210.13344v1 [cs.CL] 24 Oct 2022

Domain Example Queries

Food-Order Give me three

| {z }

quantity

large

| {z }

size

burgers

| {z }

plus item

and two

|{z}

quantity

fries

|{z}

plus item

Can I get a burrito bowl

| {z }

plus item

with brown rice

| {z }

plus item

and black beans

| {z }

plus item

, extra chicken

| {z }

plus item

and no tomatoes

| {z }

minus item

Gaming I’d like to see your ﬁre

|{z}

enchantment

swords

| {z }

item

and shields

| {z }

item

I’d like to see your ﬁre

|{z}

enchantment

swords

| {z }

item

and a shield

| {z }

item

Stocks Show me all the healthcare

| {z }

sector

companies in Europe

| {z }

location inside

outside of Germany

| {z }

location outside

Which companies have a 2018

| {z }

date

market cap

| {z }

metric name

over a million

| {z }

ﬁlter amount above

dollars and 2019

| {z }

date

revenue

| {z }

metric name

less than 2 million

| {z }

ﬁlter amount below

Table 1: Examples of challenging utterances across three different domains that contain relations between slots. The output

of classiﬁcation and slot-ﬁlling models is not sufﬁcient to correctly handle the examples above, and without incorporating

a statistical relation extraction model, developers of a dialogue system need a workaround such as the one we describe in

Section 3.1.

scales: sentence-level, bag-level, document-level, and few-

shot (Han et al. 2019). Our task is a sentence-level task, con-

sidering one dialogue utterance at a time.

Prior work has shown a variety of techniques for learn-

ing to do the task, from hand-crafting rules (Hearst 1992),

to supervised learning (Wang et al. 2016; Miwa and Bansal

2016), and weak or distant learning (Mintz et al. 2009; Hoff-

mann et al. 2011; Zeng et al. 2015; Lin et al. 2016). Model-

ing approaches have also varied, with neural networks domi-

nating recently (Goswami et al. 2020; Christopoulou, Miwa,

and Ananiadou 2021; Zhang et al. 2019). While there is an

abundance of choices for models, to demonstrate the bene-

ﬁt of incorporating any RE model, we consider two neural

approaches, an LSTM model and a transformer.

2.2 Task-oriented Dialogue Systems

A range of architectures have been explored for task-

oriented dialogue systems. Work on modular structures such

TRINDI (Larsson and Traum 2000) and DIPPER (Bos et al.

2003) proposed ways to connect a variety of models with a

structured representation of information states. At the other

end of the spectrum, end-to-end neural approaches have

been explored (Peng et al. 2020; Ham et al. 2020; Hosseini-

Asl et al. 2020). However, most commercial conversational

platforms such as Dialog Flow, Watson, or Lex employ a

modular, constrained, approach as shown in Figure 1. We

explore an extension of this approach that can maintain the

necessary accuracy level while expanding the range of sup-

ported queries.

While we focus on relation extraction, there are many

other forms of structured annotation that have been consid-

ered for dialogue. These include hierarchical slots (Gupta

et al. 2018), dependency parses (Davidson, Yu, and Yu

2019), and abstract meaning representation (Bonial et al.

2020), such as the Alexa Meaning Representation Language

(AMRL) (Kollar et al. 2018). However, labeled dependency

accuracy is only 78%, while the AMR annotations are for

human-robot commands and AMRL obliges to a strict on-

tology. While Gupta et al. (2018)’s data is the closest to

ours, they also take an end-to-end approach for extracting

slots and intents. Finally, there has been work on systems

that are domain general, such as the TRIPS parser (Allen

et al. 2018). These have been applied to biomedical text

and blocks world experiments, but not to the task-oriented

setting we consider, and doing so would require substantial

work to map the domain-general ontology to a speciﬁc need.

2.3 Conversational Semantic Parsing

Recently, researchers have turned to applying semantic pars-

ing to build dialogue representations (Aghajanyan et al.

2020; Cheng et al. 2020; Andreas et al. 2020). Unlike mod-

ular systems that represent user utterances using intent and

slot information, conversational semantic parsing builds pro-

grammic representations directly from dialogue. The re-

sulting compositional representations allow researchers to

tackle utterances that contain complex relational informa-

tion. While these representations become much richer, build-

ing and collecting programmic annotations of dialogue,

which can sometimes contain application speciﬁc APIs, also

becomes harder. Our work strives to achieve a step towards

the best of both worlds – a compositional yet simple repre-

sentation of user utterances.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AugmentingTask-OrientedDialogueSystemswithRelationExtractionAndrewLee,1ZhenguoChen,2KevinLeach,3JonathanKummerfeld11UniversityofMichigan2Bloomberg3VanderbiltUniversityajyl@umich.edu,Zhenguo.Chen@colorado.edu,kevin.leach@vanderbilt.edu,jkummerf@umich.eduAbstractThestandardtask-orienteddialoguepipelin...

展开>> 收起<<

Augmenting Task-Oriented Dialogue Systems with Relation Extraction Andrew Lee1Zhenguo Chen2Kevin Leach3Jonathan Kummerfeld1 1University of Michigan.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Augmenting Task-Oriented Dialogue Systems with Relation Extraction Andrew Lee1Zhenguo Chen2Kevin Leach3Jonathan Kummerfeld1 1University of Michigan

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: