TIARA Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases Yiheng Shu Zhiwei Yu Yuhan Li Börje F. Karlsson Tingting Ma

2025-05-06 0 0 584.78KB 14 页 10玖币

侵权投诉

TIARA: Multi-grained Retrieval for Robust Question Answering over

Large Knowledge Bases

Yiheng Shu♥∗, Zhiwei Yu♦, Yuhan Li♣∗, Börje F. Karlsson♦, Tingting Ma♠∗,

Yuzhong Qu♥and Chin-Yew Lin♦

♥State Key Laboratory of Novel Software Technology, Nanjing University, China;

♦Microsoft Research; ♣Nankai University; ♠Harbin Institute of Technology

yhshu@smail.nju.edu.cn,yzqu@nju.edu.cn,yuhanli@mail.nankai.edu.cn

{zhiwei.yu,borje.karlsson,cyl}@microsoft.com

Abstract

Pre-trained language models (PLMs) have

shown their effectiveness in multiple scenar-

ios. However, KBQA remains challenging,

especially regarding coverage and generaliza-

tion settings. This is due to two main factors:

i) understanding the semantics of both ques-

tions and relevant knowledge from the KB;

ii) generating executable logical forms with

both semantic and syntactic correctness. In

this paper, we present a new KBQA model,

TIARA, which addresses those issues by ap-

plying multi-grained retrieval to help the PLM

focus on the most relevant KB contexts, viz.,

entities, exemplary logical forms, and schema

items. Moreover, constrained decoding is used

to control the output space and reduce gen-

eration errors. Experiments over important

benchmarks demonstrate the effectiveness of

our approach. TIARA outperforms previous

SOTA, including those using PLMs or oracle

entity annotations, by at least 4.1 and 1.1 F1

points on GrailQA and WebQuestionsSP, re-

spectively. Speciﬁcally on GrailQA, TIARA

outperforms previous models in all categories,

with an improvement of 4.7 F1 points in zero-

shot generalization.1

1 Introduction

Knowledge base question answering (KBQA) has

established itself as an important and promising re-

search area as it greatly helps the accessibility and

usability of existing large-scale knowledge bases

(KBs). Such KBs contain abundant facts in a struc-

tured form, which can not only be accessed, but

also reasoned over. KBQA bypasses the need for

users to learn complex and burdensome formal

query languages, and it allows users to leverage

knowledge using natural language. While recent

KBQA efforts have achieved interesting results,

Code is available at https://github.com/microsoft/KC/

tree/main/papers/TIARA.

*Work performed during their internships at Microsoft

Research Asia.

most previous studies target or assume an under-

lying strong correspondence between the distribu-

tions of schema items in test questions and training

data, i.e., the i.i.d. assumption. However, this as-

sumption does not hold, especially for large-scale

KBs, which typically contain very large numbers of

entities and schema items (classes and relations).

This extensive space is a critical challenge as it

both allows for a myriad of novel compositions of

schema items (Keysers et al.,2020), i.e., requires

compositional generalization. It also immensely in-

creases the likelihood of user queries, including pre-

viously unseen items or domains (Gu et al.,2021),

i.e., requires strong zero-shot generalization.

Meanwhile, pre-trained language models

(PLMs), such as T5 (Raffel et al.,2020) and GPT-3

(Brown et al.,2020), have demonstrated notable

success in many natural language processing

(NLP) scenarios (Karpas et al.,2022). Many times

they even show strong generalization capabilities.

Inspired by the progress of PLMs on unstructured

text-to-text tasks, researchers have explored

semantic parsing of natural language to logical

form utilizing PLMs (Poesia et al.,2022;Xie et al.,

2022) to tackle the generalization challenges and

improve system robustness. However, differently

from unstructured data in the typical PLM pre-

training phase, KBs represent rich semantics and

complex structures, which lead to two challenges

in using PLMs for KBQA: i)

KB Grounding

given a KB, how to understand the semantics of

both the question and the relevant knowledge

from the KB? ii)

Logical Form Generation

: how

to make sure syntax and semantics of generated

logical forms conform to the KB speciﬁcation and

are executable (i.e., guarantee correctness)?

KB grounding requires linking a question to rel-

evant KB items, but the large size of KBs makes it

Our experiments involve more than 45M entities, 2K

classes, and 6K relations. RDF Schema contains rdfs:Class

(class) and rdf:Property (relation).

arXiv:2210.12925v1 [cs.CL] 24 Oct 2022

Question what napa county wine is 13.9 percent alcohol by volume?

(AND wine.wine (AND (JOIN (R wine.wine_sub_region.wines) m.0l2l_)

(JOIN wine.wine.percentage_alcohol 13.9^^float)))

Schema Retrieval

Entity Retrieval

Mention Detection napa county

Candidate Generation (napa valley, m.0l2l_) (napa airport, m.0dlb8x)

Entity Disambiguation m.0l2l_ m.0dlb8x

Exemplary Logical Form Retrieval

(AND wine.wine (JOIN wine.wine.percent_new_oak 13.9^^float))

(AND wine.wine (JOIN wine.wine.wine_sub_region m.0l2l_))

…

Class

wine.wine

wine.wine_type

wine.vineyard

wine.wine_region

food.beverage

wine.wine_color

…

Relation

wine.wine.percentage_alcohol

wine.wine_region

wine.wine_region.wines

wine.wine_sub_region.wines

wine.wine_wine_sub_region

wine.wine_country

…

Class Trie Relation Trie

Decoding Constraints PLM

Figure 1: Overview of TIARA. 1) Entity retrieval grounds the mention to entity m.0l2l_. 2) Exemplary logical

form retrieval enumerates logical forms starting from the entity m.0l2l_ or the number 13.9, and ranks them. 3)

Schema retrieval independently grounds the most related schema items. 4) Retrieved multi-grained contexts are

then fed to the PLM for generation. 5) Constrained decoding controls the schema search space during logical form

generation.

challenging. We consider that contextual retrieval

can help the PLM focus on the most relevant con-

texts (Xie et al.,2022;Wei et al.,2022). Previous

works propose different retrieval methods. Das

et al. (2021) and Zan et al. (2022) retrieve ques-

tions with logical form annotations that are similar

to the input question from the training set, which is

only effective for i.i.d. scenarios. Ye et al. (2021)

retrieve exemplary logical forms only within a two-

hop range of linked entities. Chen et al. (2021) and

Xie et al. (2022) retrieve schema items for each

question without considering the connectivity on

the KB and have poor zero-shot performance.

To make the best of structured KB contexts and

be able to take advantage of recent PLM work, we

analyze all stages in a KBQA system to identify

and address their challenges, and we demonstrate

how leveraging multi-grained retrieval augments

PLM generation - especially regarding robustness

in compositional and zero-shot settings: (i)

En-

tity retrieval

: ﬁnding topic entities is a key step,

and we augment the mention detection method to

improve entity retrieval (linking) performance in

zero-shot cases; (ii)

Exemplary logical form re-

trieval

: logical forms provide the semantic and

structural contexts from the KB, which assists the

PLMs in KB grounding and in generating valid

logical forms; and (iii)

Schema retrieval

: logical

form enumeration alone cannot properly support

questions with more than two hops or diverse func-

tion types. However, retrieving schema items is not

subject to this constraint and it can be used as a

semantic supplement to logical forms.

Although KB contexts are retrieved for the PLM,

it may still generate invalid logical forms due to

its unconstrained output space, e.g., generating

schema items that do not exist in the KB. Similarly

to how seq2seq models combined with rules (Liang

et al.,2017;Chen et al.,2021) reduces syntax errors

during logical form generation, we further intro-

duce

constrained decoding

to alleviate this issue

for the PLM using preﬁx trees constructed from the

KB.

Here we propose a mul

-gr

ined

etrieval

ugmented (

TIARA

) KBQA method (Section 3)

that addresses the two mentioned challenges. As

shown in Figure 1, we employ multi-grained re-

trieval to provide both semantic and syntactic refer-

ences to a PLM. Then, target logical form genera-

tion is controlled by constrained decoding. Utiliz-

ing these mechanisms, TIARA achieves improved

KBQA performance not only in i.i.d., but also in

compositional and zero-shot generalization (Sec-

tion 4). Experiments over two important bench-

marks demonstrate the effectiveness of our ap-

proach. TIARA outperforms all previous SOTA

methods, including those using PLMs or even or-

acle entity annotations, by at least 4.1 and 1.1 F1

points on GrailQA and WebQuestionsSP, respec-

tively. Speciﬁcally on GrailQA, TIARA outper-

forms previous models in all generalization cat-

egories, with an improvement of 4.7 F1 points

in zero-shot generalization. To further explore

how TIARA strengthens the potential of PLMs for

KBQA, we also analyze the impact of providing

PLMs with different granularities of contexts on

different generalization levels and question types

(Section 5).

2 Related Work

2.1 Knowledge Base Question Answering

KBQA endeavors to allow querying KBs through

natural language questions. Existing KBQA meth-

ods are mainly either information retrieval-based

(IR-based) or semantic parsing-based (SP-based)

methods (Lan et al.,2021). IR-based methods

(Shi et al.,2021;Zhang et al.,2022) construct a

question-speciﬁc graph and rank entities or paths

to get the top answers. SP-based methods aim at

generating target logical forms, which makes them

more interpretable. Such methods can be classiﬁed

into feature-based ranking and seq2seq generation

methods. Feature-based ranking methods search

and rank query graphs in a step-wise manner. Yih

et al. (2015) deﬁne a query graph and propose a

staged generation method. Lan and Jiang (2020) in-

corporate constraints during the staged query graph

generation. Hu et al. (2021) use rules to decom-

pose questions and guide the search. Seq2seq gen-

eration methods (Liang et al.,2017;Chen et al.,

2021) convert natural language sequences to log-

ical form sequences. They are more ﬂexible than

the above ranking methods in generating logical

forms with more hops or functions. In particular,

seq2seq generation methods using PLMs (Oguz

et al.,2020;Das et al.,2021;Ye et al.,2021;Cao

et al.,2022a;Hu et al.,2022) show promising per-

formance. Among them, the most relevant work to

ours is RnG-KBQA (Ye et al.,2021), which also

retrieves exemplary logical forms as important con-

texts. However, TIARA further explores the use

of PLMs by improving zero-shot mention detec-

tion, matching entity-independent semantics with

schema retriever, and reducing generation errors

with constrained decoding.

2.2 Neural Semantic Parsing with PLMs

Several existing neural semantic parsing methods

consider contextual retrieval when applying PLMs.

Shi et al. (2022) use nearest neighbor retrieval to

augment zero-shot inference. Xie et al. (2022)

feed contexts to PLMs and conduct experiments

on several semantic parsing tasks over structured

data. Furthermore, other works also leverage con-

strained decoding for semantic parsing. Scholak

et al. (2021) propose multiple levels of constrained

decoding for the text-to-SQL task and improve per-

formance without additional training. Shin et al.

(2021) illustrate PLMs with constrained decoding

have few-shot semantic parsing ability. While, Poe-

sia et al. (2022) propose that retrieval of related

examples and constrained decoding can improve

the performance of the code generation task with-

out ﬁne-tuning. In this paper, we introduce the idea

of multi-grained contextual retrieval along with

constrained decoding to bolster KBQA scenarios.

3 Method

To demonstrate how comprehensive KB context

enhances the robustness of retrieval, a key com-

ponent of our method, we retrieve KB contexts in

a multi-grained manner and then generate target

logical forms with constrained decoding, as shown

in Figure 1. We introduce multi-grained retrieval

in Section 3.2, target logical form generation in

Section 3.3, and describe constrained decoding in

Section 3.4.

3.1 Preliminaries

Here, a knowledge base is an RDF graph consist-

ing of a collection of triples in the form (

where

is an entity,

is a relation, and

can be

an entity, a literal, or a class. We use s-expressions

as the logical form following Gu et al. (2021) and

Ye et al. (2021), which can later be converted to

SPARQL queries and executed over KBs. In this

paper,

exemplary logical form

refers to the logi-

cal form that is input to the generative PLM as the

context. It is obtained by enumeration and ranking

on KBs.

Schema

refers to rdfs:Class (class) and

rdf:Property (relation) together, and they are the

necessary elements of logical forms.

Target log-

ical form

refers to the logical form generated by

the PLM. Examples of above concepts are shown

in Figure 1.

3.2 Multi-grained Retriever

Entity Retriever

We perform entity retrieval fol-

lowing a standard pipeline with three steps, i.e.,

mention detection, candidate generation, and entity

disambiguation (Shen et al.,2021). To better detect

zero-shot mentions, we regard mention detection

as a span classiﬁcation task (Jiang et al.,2021). In

this way, we can deal well with sentences contain-

ing out-of-vocabulary entities (Fu et al.,2021) and

achieve higher recall by adjusting the threshold of

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

TIARA:Multi-grainedRetrievalforRobustQuestionAnsweringoverLargeKnowledgeBasesYihengShu~,ZhiweiYu},YuhanLi|,BörjeF.Karlsson},TingtingMa,YuzhongQu~andChin-YewLin}~StateKeyLaboratoryofNovelSoftwareTechnology,NanjingUniversity,China;}MicrosoftResearch;|NankaiUniversity;HarbinInstituteofTechnologyyh...

展开>> 收起<<

TIARA Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases Yiheng Shu Zhiwei Yu Yuhan Li Börje F. Karlsson Tingting Ma.pdf

共14页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

TIARA Multi-grained Retrieval for Robust Question Answering over Large Knowledge Bases Yiheng Shu Zhiwei Yu Yuhan Li Börje F. Karlsson Tingting Ma

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: