Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge Kaixin May Hao Cheng Xiaodong Liu Eric Nyberg Jianfeng Gao

2025-05-02 0 0 681.01KB 15 页 10玖币

侵权投诉

Open-domain Question Answering via Chain of Reasoning over

Heterogeneous Knowledge

Kaixin Ma♣†∗, Hao Cheng♠∗, Xiaodong Liu♠, Eric Nyberg♣, Jianfeng Gao♠

♣Carnegie Mellon University ♠Microsoft Research

{kaixinm,ehn}@cs.cmu.edu {chehao,xiaodl,jfgao}@microsoft.com

Abstract

We propose a novel open-domain question

answering (ODQA) framework for answer-

ing single/multi-hop questions across hetero-

geneous knowledge sources. The key novelty

of our method is the introduction of the in-

termediary modules into the current retriever-

reader pipeline. Unlike previous methods that

solely rely on the retriever for gathering all ev-

idence in isolation, our intermediary performs

a chain of reasoning over the retrieved set.

Speciﬁcally, our method links the retrieved

evidence with its related global context into

graphs and organizes them into a candidate list

of evidence chains. Built upon pretrained lan-

guage models, our system achieves competi-

tive performance on two ODQA datasets, OTT-

QA and NQ, against tables and passages from

Wikipedia. In particular, our model substan-

tially outperforms the previous state-of-the-art

on OTT-QA with an exact match score of 47.3

(45 %relative gain).

1 Introduction

The task of open-domain question answering

(ODQA) typically involves multi-hop reasoning,

such as ﬁnding relevant evidence from knowledge

sources, piecing related evidence with context to-

gether, and then producing answers based on the

ﬁnal supportive set. While many questions can

be answered by a single piece of evidence (Joshi

et al.,2017;Kwiatkowski et al.,2019), answering

more complex questions are of great interest and

require reasoning beyond local document context

(Yang et al.,2018;Geva et al.,2021). The problem

becomes more challenging when evidence is scat-

tered across heterogeneous sources, e.g., unstruc-

tured text and structured tables (Chen et al.,2020a),

which necessitates hopping from one knowledge

modality to another. Consider the question in Fig. 2.

†

Part of this work is done during an internship at Mi-

crosoft Research

∗Equal contribution

Figure 1: Our CORE vs. previous retriever-only methods

for evidence discovery. The grey square is the question,

the blue dots are 1st-hop passages, the blue triangles

are 1st-hop tables, and the purple dots are 2nd-hop pas-

sages.

To form the ﬁnal answer, a model needs to ﬁnd the

entry list (a table) of the mentioned touring car

race, look up the driver name with the correct rank,

search for the corresponding driver information,

and extract the birthplace from the free-form text.

Existing retriever-reader methods (Min et al.,

2021,inter alia) for ODQA mainly customize the

retriever model for tackling individual question

types, i.e., exclusively relying on the retriever to

gather all necessary context in a query-dependent

fashion. As shown in Fig. 1, the single-hop model

(Karpukhin et al.,2020) only retrieves a list of iso-

lated passages (blue dots). For multi-hop cases, an

iterative retriever looks for a query-dependent path

of passages (blue-purple dot chains) (Xiong et al.,

2020), i.e., the later hop passages are retrieved us-

ing expanded queries including the original ques-

tion and previous hop passages. Although those

retrieval-only methods achieve promising results

on their targeted cases, the customized retrievers

are unable to generalize well. For example, an iter-

ative passage retriever trained with both multi-hop

and single-hop questions performs poorly over both

arXiv:2210.12338v1 [cs.CL] 22 Oct 2022

Figure 2: An illustration of CORE for ODQA. Given a question, the retriever ﬁrst ﬁnds hop-1 evidence from the

entire Wikipedia (orange arrows). Then the linker gathers relevant documents for the hop-1 evidence (purple

arrows), which are treated as hop-2 evidence. Next, the chainer reranks all hop-1 and hop-2 evidence and splices

them together into evidence chains (blue arrows). Finally, the reader takes in the top-50 chains and produces the

answer (black arrows). The gold evidence chain is marked in red.

types (Xiong et al.,2020). For real-world applica-

tions with heterogeneous knowledge sources, it is

desirable for an ODQA system to handle both cases

well, and the retrieval-only methods are unlikely to

succeed.

We propose a novel

hain

asoning (

CORE

)

framework that generalizes well on both single-hop

and multi-hop question answering. The main con-

tribution of

CORE

is the introduction of two inter-

mediary modules, the linker and the chainer, that

play the bridging role between the retriever and the

reader, i.e., piecing together related evidence with

necessary context for single/multi-hop questions.

These two modules work in a forward-backward

fashion. In the forward pass, the linker, a novel

table entity linking model (§3.1), links the raw evi-

dence with its related context (e.g., table-passage

graphs in Fig. 1). The chainer, a new unsuper-

vised reranker (§3.2), then prunes all linked evi-

dence using the corresponding question generation

scores from a pretrained language model (Sanh

et al.,2021) to form a shortlist of relevant evidence

chains in a backward noisy channel fashion (e.g.,

table-passage paths in Fig. 1). By delegating the

hopping operation to the intermediary, our formal-

ization can potentially gather evidence more effec-

tively over different question types.

To demonstrate the effectiveness of

CORE

, we

evaluated the proposed model on two popular

ODQA datasets, the multi-hop dataset OTT-QA

(Chen et al.,2020a) and the single-hop dataset NQ

(Kwiatkowski et al.,2019). Empirically, we show

that our approach is general for both types of rea-

soning in ODQA. In particular,

CORE

substantially

outperforms the previous state-of-the-art (SOTA)

on OTT-QA by 14+ points on exact match scores

(

45%+

relative gain), and it is competitive with

SOTA models on NQ. Moreover, we show that

one single uniﬁed model can learn to solve both

tasks under our framework. From our analysis, we

also ﬁnd that our evidence chains can potentially

help answer single-hop questions by enriching the

evidence with more supportive context. 1

2 Overview of the CORE Framework

The

CORE

framework is designed to answer ques-

tions where the answer is a contiguous span from

a table

or a passage

. Here neither

nor

given, so they need to be retrieved from the table

corpus

and the passage corpus

, respectively.

For single-hop questions, a single

may be

sufﬁcient, whereas for multi-hop questions, one or

more tand pare required to ﬁnd the answer.

As shown in Fig. 2,

CORE

consists of a retriever,

alinker, a chainer and a reader. We adopt the DPR

model (Karpukhin et al.,2020) as our retriever.

We only brieﬂy describe the retriever here as it is

not the main focus of our work. The DPR is a

bi-encoder model that consists of a question en-

coder and a context encoder. For our setting, the

questions and passages/tables are represented by

the

[CLS]

embedding produced by their respective

encoder, and the retrieval is done by maximum

inner product search in the vector space. For a

given question, we use DPR to retrieve the initial

evidence set which includes tables and passages.

Data and code available at

https://github.com/

Mayer123/UDT-QA

Given the initial evidence set (e.g., the car race

entry list table in Fig. 2), our intermediary module

produces a list of query-dependent evidence chains

(e.g., the red line linked evidence chain consisting

of the car race entry list and the driver’s Wikipedia

page). We ﬁrst propose a linker model (§3.1) to ex-

pand the candidate evidence set by including extra

passages related to tables in the initial set (purple

arrows in Fig. 2). This step allows the model to

enrich the evidence context, especially including

reasoning chains needed for multi-hop questions.

Since there could be many links between a piece

of evidence and others (i.e., a densely connected

graph), considering all links is computationally in-

feasible for the downstream reader. Thus, we de-

velop a chainer model (§3.2) to prune the evidence

graph with the corresponding question and then

chain the evidence across hops together to form

query-dependent paths. Here, we only keep top-

scored chains for reading so that the reader can

work on a ﬁxed computation budget.

Finally, the Fusion-in-Decoder (FiD) (Izacard

and Grave,2021), a T5-based generative model

(Raffel et al.,2019), is used as the reader for gen-

erating the ﬁnal answer. The model ﬁrst encodes

each top-

evidence chain independently along

with the question. During decoding, the decoder

can attend to all chains, thus fusing all the input

information.

3 Intermediary Modules

In this part, we present the two key components of

CORE

for supporting multi-hop reasoning, i.e., the

linker for building evidence graphs and the chainer

for forming query-dependent paths.

3.1 Linker

In this work, we mainly focus on linking an en-

tity mention in the retrieved evidence to the cor-

responding Wikipedia page for building evidence

graphs. This setup is related to the recent entity

linking work (Wu et al.,2020). However, there are

important modiﬁcations for ODQA. In particular,

instead of assuming the entity mention as a prior,

we consider a more realistic end-to-end scenario for

ODQA: the linker model has to ﬁrst propose candi-

date entity mentions (spans) for a given evidence

(e.g., “Tony Longhurst” in Fig. 2), and then links

the proposed mention to its Wikipedia page. An-

other major difference is that we study entity men-

tions in tables instead of text. As tables contain

more high-level summary information than text,

using tables as pivots for constructing evidence

graphs can potentially help improve the recall of

evidence chains for QA. In the meanwhile, this

task is challenging due to the mismatch between

the lexical form of the table cells and their linked

passage titles. For example, the table of "NCAA Di-

vision I women’s volleyball tournament" contains

the cell VCU, which refers to VCU Rams instead of

Virginia Commonwealth University. Thus simple

lexical matching would not work.

In the following, we ﬁrst describe the model

for entity mention proposal and then present a

novel entity linking model for mentions in tables.

Both models are based on a pretrained language

model, BERT (Devlin et al.,2019). Following

previous work (Oguz et al.,2020), we ﬂatten the

table row-wise into a sequence of tokens for deriv-

ing table representations from BERT. In particular,

we use

x1, . . . , xN

to denote an input sequence of

length

. Typically, when using BERT, there is a

prepended token

[CLS]

for all input sequences, i.e.,

[CLS], x1, . . . , xN

. Then the output is a sequence

of hidden states

h[CLS],h1,...,hN∈Rd

from the

last BERT layer for each input token, where

the hidden dimension.

Entity Mention Proposal

In realistic settings, the

ground truth entity mention locations are not pro-

vided. Directly applying an off-the-shelf named

entity recognition (NER) model can be sub-optimal

as the tables are structured very differently from the

text. Thus, we develop a span proposal model to

label the entity mentions in the table. Speciﬁcally,

we use BERT as the encoder (BERT

) and add a

linear projection to predict whether a token is part

of an entity mention for all tokens in the table,

hm1,...,hmN=BERT m(t1, . . . , tN),(1)

y=Whm,(2)

where

hm∈RN×d

and

W∈R2×d

. The model is

trained with a token-level binary loss

−1

n=1

(ynlog P(ˆ

y)1+ (1 −yn) log P(ˆ

y)0),

(3)

where

is the 0-1 label for the token at position

n, and P(·)is the softmax function.

Table Entity Linking

Once the candidate entity

mentions are proposed, we follow Wu et al. (2020)

to use a bi-encoder model for linking. Simi-

larly, two BERT models are used to encode ta-

bles (BERT

) and passages (BERT

), respectively.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Open-domainQuestionAnsweringviaChainofReasoningoverHeterogeneousKnowledgeKaixinMa|y,HaoCheng,XiaodongLiu,EricNyberg|,JianfengGao|CarnegieMellonUniversityMicrosoftResearch{kaixinm,ehn}@cs.cmu.edu{chehao,xiaodl,jfgao}@microsoft.comAbstractWeproposeanovelopen-domainquestionanswering(ODQA)framewor...

展开>> 收起<<

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge Kaixin May Hao Cheng Xiaodong Liu Eric Nyberg Jianfeng Gao.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge Kaixin May Hao Cheng Xiaodong Liu Eric Nyberg Jianfeng Gao

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: