Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder Patrick Huber and Giuseppe Carenini

2025-05-06 0 0 239.26KB 3 页 10玖币

侵权投诉

Unsupervised Inference of Data-Driven Discourse Structures

using a Tree Auto-Encoder

Patrick Huber and Giuseppe Carenini

Department of Computer Science

University of British Columbia

Vancouver, BC, Canada, V6T 1Z4

{huberpat, carenini}@cs.ubc.ca

1 Introduction

Discourse Parsing is a key NLP task for processing

multi-sentential natural language. Most research

in the area thereby focuses on one of the two main

discourse theories – RST (Mann and Thompson,

1988) or PDTB (Prasad et al.,2008). While these

discourse theories are of great value for the ﬁeld

of discourse parsing and have been guiding its

progress ever since, there are two major problems

when data is annotated according to those theories:

(1) Due to the inherently ambiguous nature of nat-

ural language, the inter-annotator agreement when

following the annotation guidelines is relatively

low (Carlson et al.,2003) and (2) The annotation

process itself is expensive and tedious, limiting the

size of available gold-standard datasets.

With a growing need for robust and general dis-

course structures in many downstream tasks and

real-world applications (e.g. Gerani et al. (2014);

Nejat et al. (2017); Ji and Smith (2017)), the cur-

rent lack of high-quality, high-quantity discourse

trees poses a severe shortcoming. Fortunately,

there are more data-driven alternatives to infer dis-

course structures. For example, the recently pro-

posed MEGA-DT treebank (Huber and Carenini,

2019), whose discourse structures and nuclearity at-

tributes are automatically inferred from large-scale

sentiment datasets, reaching state-of-the-art perfor-

mance on the inter-domain discourse parsing task.

Similarly, another approach by Liu et al. (2019) in-

fers latent discourse trees from the downstream

summarization task using a transformer model.

Outside the area of discourse parsing, syntactic

trees have previously been inferred according to

several diverse strategies, e.g. Socher et al. (2011);

Yogatama et al. (2016); Choi et al. (2018); Maillard

et al. (2019). In general, the approaches mentioned

above have been shown to capture valuable infor-

mation. Some models outperform baselines trained

on human datasets (see Huber and Carenini (2019)),

others are proven to enhance diverse downstream

tasks (Liu et al.,2019;Choi et al.,2018). How-

ever, despite these initial successes, one limitation

that all aforementioned models share is the task-

speciﬁcity, oftentimes only capturing downstream

task-related components and potentially compro-

mising the generality of the resulting trees, as for

instance shown for the model using summarization

data (Liu et al.,2019)inFerracane et al. (2019).

In order the alleviate the limitation described

above, we propose a new strategy to generate tree

structures in a task-agnostic, unsupervised fash-

ion by extending a latent tree induction framework

(Choi et al.,2018) with an auto-encoding objec-

tive. The proposed approach can be applied to any

tree-structured objective, such as syntactic parsing,

discourse parsing and others. However, due to the

especially difﬁcult annotation process to generate

discoursetrees, we initially intend to develop such

method to complement task-speciﬁc models in gen-

erating much larger and more diverse discourse

treebanks.

2 Unsupervised Tree Auto-Encoder

Our intuition for the tree auto-encoder comes from

previous work, indicating that with available gold-

standard trees, the programming-language transla-

tion task can learn valid projections in a tree-to-tree

style neural network (Chen et al.,2018). Further-

more, variational auto-encoders have been shown

effective for the difﬁcult task of grammar induc-

tion (Kusner et al.,2017). Generally speaking, our

new model is similar in spirit to the commonly

used sequence-to-sequence (seq2seq) architecture

(Sutskever et al.,2014), which has also been in-

terpreted as a sequential auto-encoder (Li et al.,

2015). However, our approach generalizes on the

seq2seq model, which represents a special (left-

arXiv:2210.09559v1 [cs.CL] 18 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

UnsupervisedInferenceofData-DrivenDiscourseStructuresusingaTreeAuto-EncoderPatrickHuberandGiuseppeCareniniDepartmentofComputerScienceUniversityofBritishColumbiaVancouver,BC,Canada,V6T1Z4fhuberpat,careninig@cs.ubc.ca1IntroductionDiscourseParsingisakeyNLPtaskforprocessingmulti-sententialnaturallanguag...

展开>> 收起<<

Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder Patrick Huber and Giuseppe Carenini.pdf

共3页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder Patrick Huber and Giuseppe Carenini

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: