Cross-domain Generalization for AMR Parsing Xuefeng Bai Sen Yang Leyang Cui Linfeng Song Yue Zhangy School of Engineering Westlake University China

2025-05-06 0 0 623.47KB 16 页 10玖币

侵权投诉

Cross-domain Generalization for AMR Parsing

Xuefeng Bai∗∗♠ , Sen Yang♥, Leyang Cui♣, Linfeng Song♦, Yue Zhang♠†

♠School of Engineering, Westlake University, China

♥The Chinese University of Hong Kong, China

♣Tencent AI Lab, Shenzhen, China

♦Tencent AI Lab, Bellevue, WA, USA

†Institute of Advanced Technology, Westlake Institute for Advanced Study, China

Abstract

Abstract Meaning Representation (AMR) pars-

ing aims to predict an AMR graph from textual

input. Recently, there has been notable growth

in AMR parsing performance. However, most

existing work focuses on improving the per-

formance in the speciﬁc domain, ignoring the

potential domain dependence of AMR parsing

systems. To address this, we extensively

evaluate ﬁve representative AMR parsers on

ﬁve domains and analyze challenges to cross-

domain AMR parsing. We observe that chal-

lenges to cross-domain AMR parsing mainly

arise from the distribution shift of words and

AMR concepts. Based on our observation,

we investigate two approaches to reduce the

domain distribution divergence of text and

AMR features, respectively. Experimental

results on two out-of-domain test sets show the

superiority of our method.

1 Introduction

Abstract meaning representation (AMR; Banarescu

et al. 2013) is a broad-coverage semantic structure

formalism that represents the meaning of a text in

a rooted directed graph. As shown in Figure 1, the

nodes in an AMR graph represent concepts such

as entities and predicates, and the edges indicate

their semantic relations. AMR parsing (Flanigan

et al.,2014;Konstas et al.,2017;Lyu and Titov,

2018;Guo and Lu,2018;Zhang et al.,2019a;Cai

and Lam,2020a;Bevilacqua et al.,2021;Zhou

et al.,2021b;Bai et al.,2022a) is the task of

transforming natural language into AMR graphs.

This is a fundamental task in semantics, which can

also beneﬁt downstream use.

AMR has been proven to be useful for

many downstream tasks, such as information

extraction (Huang et al.,2016;Martínez-Rodríguez

et al.,2020;Zhang and Ji,2021;Luo et al.,

2022;Chen et al.,2022b;Wang et al.,2022), text

∗Work done as an intern at Tencent AI Lab Seattle.

police

hum-02

:arg0

boy

:beneficiary

walk-01

:time

:arg0

town

:destination

Figure 1: An AMR graph for sentence “The police

hummed to the boy as he walked to town.”

summarization (Liu et al.,2015;Liao et al.,2018;

Chen et al.,2021,2022c;He et al.,2022), machine

translation (Song et al.,2019;Slobodkin et al.,

2022;Chen et al.,2022a), text generation (Konstas

et al.,2017;Song et al.,2018;Zhu et al.,

2019;Bai et al.,2020;Ribeiro et al.,2021),

and dialogue systems (Bonial et al.,2020;Bai

et al.,2021,2022b). To beneﬁt such a diverse

set of tasks that covers various domains, an

ideal AMR parser should generalize well across

different domains. However, most existing work

only focuses on improving the in-domain parsing

accuracy, ignoring the performances on other

domains. Though state-of-the-art AMR parsers

can obtain a SMATCH score of over

84%

on an in-

domain test set, we observe that their cross-domain

performance is still weak (e.g., lower than

65%

on the biomedical domain). It remains an open-

question how well different types of AMR parsers

generalize to out-of-domain (OOD) data.

In this work, we take the ﬁrst step to study

the cross-domain generalization ability of a range

of typical AMR parsers, investigating three main

research questions: 1) how well do different AMR

parsers perform on out-of-domain test sets? 2)

what are the main challenges to cross-domain AMR

parsing? and 3) how to improve the performance

of cross-domain AMR parsing?

We empirically choose ﬁve major AMR parsers

for comparison, including a two-stage statistical

arXiv:2210.12445v1 [cs.CL] 22 Oct 2022

parser (Flanigan et al.,2014), a graph-based

parser (Cai and Lam,2020b), a transition-based

parser (Zhou et al.,2021b), a Seq2Seq-based

parser (Bevilacqua et al.,2021), and an AMR-

speciﬁc pre-training parser (Bai et al.,2022a). The

test domains cover news, biomedical, novel, and

wiki questions. We conduct experiments under

the zero-shot setting, where a model is trained

on the source domain and evaluated on the target

domain without using any target-domain labeled

data. Our results show that 1) all models give

relatively lower (up to 45.5%) performances on

out-of-domain test sets, with the most dramatic

drop on named entities and wiki links; 2) the graph

pretraining-based parser is stronger in domain

transfer than the other parsers; 3) the transition-

based parser is more robust than the seq2seq-

based parser. We further analyze the impact of a

set of linguistic features, and the results suggest

that the performance degradation is positively

correlated with the distribution shifts of words and

AMR concepts. Compared with the distribution

divergences of the input features, those of the

output features are more challenging to cross-

domain AMR parsing.

Based on our analysis, we investigate two

approaches to bridge the domain gap for improving

cross-domain AMR parsing. We ﬁrst continually

pre-train a BART model on target domain raw

text to reduce the distribution gap of words.

To further bridge the domain gap of output

features, we adopt a pre-trained AMR parser

to construct silver AMR graphs on the target

domain, which potentially reduces the output

features divergence. Experimental results show

that the proposed methods consistently improve

the parsing performance on out-of-domain test

sets. To our knowledge, this is the ﬁrst systematic

study on cross-domain AMR parsing. Our code

and results will be available at

https://github.com/

goodbai-nlp/AMR-DomainAdaptation.

2 Related Work

2.1 AMR Parsing

On a coarse-grained level, the current AMR parsing

systems can be categorized into two main classes.

The ﬁrst is two-stage parsing system, which ﬁrst

identiﬁes concepts, and then predicts relations

based on the concept decisions. Two tasks are

modeled either in a pipeline (Flanigan et al.,

2014,2016) or jointly (Lyu and Titov,2018;

Zhang et al.,2019a). The other one is one-

stage parsing, which generates a parse graph

incrementally. The one-stage parsing methods

can be further divided into three categories:

graph-based parsing, transition-based parsing, and

seq2seq-based parsing. Transition-based parsing

induces an AMR graph by predicting a sequence

of transition actions. The transition-based AMR

parsers either maintain a stack and a buffer (Wang

et al.,2015;Damonte et al.,2017;Ballesteros and

Al-Onaizan,2017;Vilares and Gómez-Rodríguez,

2018;Liu et al.,2018;Naseem et al.,2019;

Fernandez Astudillo et al.,2020;Lee et al.,2020)

or make use of a pointer (Zhou et al.,2021a,b).

Graph-based parsing builds a semantic graph

incrementally. At each time step, a new node

along with its connections to existing nodes are

jointly decided. The graph is induced either in top-

down manner (Cai and Lam,2019) or in speciﬁc

traversal order (Zhang et al.,2019b;Cai and Lam,

2020a). Seq2seq-based parsing treats AMR parsing

as a sequence-to-sequence problem by linearizing

AMR graphs so that existing seq2seq models can be

readily utilized. Various seq2seq architectures have

been employed for AMR parsing, such as vanilla

seq2seq (Barzdins and Gosko,2016;Konstas et al.,

2017), supervised attention (Peng et al.,2017),

character-based (Van Noord and Bos,2017), and

pre-trained Transformer (Bevilacqua et al.,2021;

Bai et al.,2022a).

Despite great success, most previous work

on AMR parsing focuses on the in-domain

setting, where the training and test data share

the same domain. In contrast, we systematically

evaluate the model performance on

out-of-

domain datasets. To our knowledge, we are

the ﬁrst to systematically study cross-domain

generalization for AMR parsing.

2.2 Related Tasks

We summarize recent research studying other

semantic formalisms as well as the cross-domain

generalization of named entity recognition (NER),

semantic role labeling (SRL) and constituency

parsing.

Semantic parsing on other formalisms. AMR is

strong-correlated with other semantic formalisms

such as semantic dependency parsing (SDP, Oepen

et al.,2016) and universal conceptual cogni-

tive annotation (UCCA, Abend and Rappoport,

2013;Hershcovich et al.,2017), and recent

The police could help the boy.

possible help-01 boypolice

:arg0

:arg1

Node Prediction

Edge Prediction

(a) Two-stage parser

The police could help the boy.

possible help-01 boypolice

:arg0

:arg1

Text Encoding

Graph Prediction

ℎ!ℎ"ℎ#ℎ$ℎ%ℎ&

Previous steps Current step

(b) Graph-based parser

The police could help the boy.

Text Encoding

Graph Prediction

ℎ!ℎ"ℎ#ℎ$ℎ%ℎ&

possible help-01

police [shift] [shift] [LA(1, :arg0)] …

The police could help the boy.

Text Encoding

Graph Prediction

ℎ!ℎ"ℎ#ℎ$ℎ%ℎ&

(⟨z!⟩possible :arg1 (⟨z"⟩help-01 :arg0 (⟨z#⟩police) :arg1 (⟨z$⟩boy))

(d) Seq2seq-based parser

Figure 2: Illustration of four AMR parsers given the input “The police could help the boy.”.

researches show that they can be represented in

a uniﬁed format and parsed by a generalized

framework (Hershcovich et al.,2018;Zhang et al.,

2019b). However, most of previous work focus on

speciﬁc domain, leaving the study of cross-domain

generalization unexplored.

Cross-domain NER

. Named entity recognition

(NER) is a subtask of AMR parsing. To build a

robust NER system across domains, Yang et al.

(2017) directly train NER models on the domain-

mixed corpus. Wang et al. (2020) introduce an

auxiliary task to predict the domain label. Recently,

many studies focus on recognizing the unseen

entity types in the target domain. Wiseman

and Stratos (2019) and Yang and Katiyar (2020)

propose distance-based methods, which copy the

entity label of nearest neighbors. Cui et al. (2021)

and Ma et al. (2021) adopt prompt-based methods

by using BART and BERT, respectively.

Cross-domain SRL

. SRL can also be seen

as semantic-related subtasks of AMR parsing.

Dahlmeier and Ng (2010) conduct an extensive

study by analyzing various features and techniques

that are used for SRL domain adaptation. Lim

et al. (2014) combine a prior model with a

structural learning model to build a multi-domain

SRL system. Do et al. (2015) exploit the

knowledge from a neural language model and

external linguistic resource for domain adaptation

on biomedical data. Rajagopal et al. (2019) develop

a label mapping strategy and a layer adapting

approach for cross-domain SRL. Compared with

cross-domain NER and SRL, the task of cross-

domain AMR parsing is more challenging since

AMR is a graph formalism, and AMR contains

more types of concepts and relations.

Cross-domain constituency parsing.

Yang et al.

(2022) investigated challenges to open-domain

syntactic parsing, introducing datasets on new

domains and analyzing the key factors on to

cross-domain constituency parsing using a set of

linguistic features. Our work is similar to their

work in studying the key challenges on various

parsing systems. However, we focus on AMR and

conducts ﬁne-grained semantic-related evaluation.

In addition, we provide a intuitive solution for

improving cross-domain AMR parsing.

3 Compared Models

We choose the representative or top-performing

parser of two-stage, graph-based, transition-based,

seq2seq-based as well as a pre-trained parser for

evaluation. In particular, the following AMR

parsing systems are considered:

JAMR

(Flanigan et al.,2014), as shown

in Figure 2(a), is a two-stage parsing model

which predicts concepts and relations in a

pipeline. JAMR identiﬁes concepts and predicts

the relations using two discriminatively-trained

linear structured predictors, which use rich features

Models Categratory Pre-proc. Post-proc. Ext. Data PLM

JAMR Two-stage X X POS, train align, etc. 7

AMRGS Graph Recat. concept, polarity, wiki POS, NER, Lemm. BERT

STRUCTBART Transition 7wiki train align. BART

SPRING Seq2seq 7wiki 7BART

AMRBART Pretrain + Seq2seq 7wiki 200k silver BART

Table 1: Compared AMR parsing systems. “Recat”–graph re-categorization.

like part-of-speech tagging (POS), named entities

recognition (NER), lemmatization, etc. In addition,

JAMR relies on an external aligner to construct

supervision signals for both stages.

AMRGS

(Cai and Lam,2020a) is a graph-based

parser which builds a semantic graph incrementally.

As shown in Figure 2(b), at every step, the graph-

based parser predicts one node and its connection to

existing graph. AMRGS learns mutual causalities

between text and graph by updating the sentence

and graph representations iteratively. AMRGS

obtains word-level representation from a pre-

trained language model (i.e., BERT (Devlin et al.,

2019)) and uses POS, NER and lemmatization as

external knowledge to make predictions.

STRUCTBART

(Zhou et al.,2021b), as shown

in Figure 2(c), is a transition-based parser which

generates an AMR graph through a sequence of

transition actions. In particular, the transition

actions are:

•SHIFT moves token cursor to right.

•<string> creates a node of name <string>.

•COPY

creates a node with the name of the

cursor-pointed token.

•LA(j, LBL)

creates an arc with label LBL from

the last generated node to the jth generated node.

•RA(j, LBL)

is same as LA but with reversed

edge direction.

•ROOT assigns the last generated node as root.

StructBART takes a pre-trained BART model as

the backbone and extends the original vocabulary

with transition actions. Additionally, StructBART

requires an external aligner to obtain oracle

transition actions for training.

SPRING

(Bevilacqua et al.,2021), as shown

in Figure 2(d), is a sequence-to-sequence parser

which transforms a text sequence into a linearized

AMR sequence. SPRING adopts a depth-ﬁrst

algorithm to transform AMR graphs into a

sequence where concepts and relations are treated

equally. To deal with co-referring nodes, SPRING

adds special tokens to the vocabulary. Same with

STRUCTBART, SPRING also initializes model

parameters with BART.

AMRBART

(Bai et al.,2022a) is a continually

pre-trained BART model on AMR graphs and

text. It uses three graph-based pre-training tasks to

improve the structure awareness of the encoder and

decoder and another four tasks that jointly learns on

text and AMR graph to capture the correspondence

between AMR and text. AMRBART is pre-trained

on 250k training instances, which lie in the same

domain as AMR2.0.

In addition, JAMR uses complicated rule-

based pre-processing and post-processing steps to

simplify the input and reconstruct the AMR graphs.

AMRGS uses rule-based graph re-categorization

for pre-processing and recovers concept sense tags,

wiki links, and polarities during post-processing.

StructBART, SPRING, and AMRBART do not

require pre-processing steps and use the BLINK

Entity Linker (Wu et al.,2020) to handle wiki links

during post-processing. Table 1summarizes the

above systems according to their characteristics.

4 Experiments

Experimental conﬁgurations and our adopted

datasets are shown in Sections 4.1 and 4.2, respec-

tively. To study the cross-domain generalization

ability of current AMR parsers, we ﬁrst quantify

the difference between in-domain training data and

out-of-domain test data (Section 4.3), and then

evaluate the cross-domain performance of

typical

AMR parsers (Section 4.4).

4.1 Experimental Settings

Model Conﬁguration

. We adopt the ofﬁcially

released code of each system and use their default

conﬁguration to re-train and evaluate the model

performance. The best model is selected according

to the performance on the in-domain validation set.

All models are trained and evaluated on a single

Nvidia Tesla V100 GPU.

Metrics.

We assess the performance of parsing

models with SMATCH (Cai and Knight,2013)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Cross-domainGeneralizationforAMRParsingXuefengBai,SenYang~,LeyangCui|,LinfengSong},YueZhangySchoolofEngineering,WestlakeUniversity,China~TheChineseUniversityofHongKong,China|TencentAILab,Shenzhen,China}TencentAILab,Bellevue,WA,USAyInstituteofAdvancedTechnology,WestlakeInstituteforAdvancedStudy,...

展开>> 收起<<

Cross-domain Generalization for AMR Parsing Xuefeng Bai Sen Yang Leyang Cui Linfeng Song Yue Zhangy School of Engineering Westlake University China.pdf

共16页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Cross-domain Generalization for AMR Parsing Xuefeng Bai Sen Yang Leyang Cui Linfeng Song Yue Zhangy School of Engineering Westlake University China

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: