ReaRev Adaptive Reasoning for Question Answering over Knowledge Graphs Costas Mavromatis andGeorge Karypis

2025-04-29 0 0 390.06KB 12 页 10玖币

侵权投诉

ReaRev: Adaptive Reasoning for Question Answering over Knowledge

Graphs

Costas Mavromatis and George Karypis

University of Minnesota, USA

{mavro016, karypis}@umn.edu

Abstract

Knowledge Graph Question Answering

(KGQA) involves retrieving entities as an-

swers from a Knowledge Graph (KG) using

natural language queries. The challenge is

to learn to reason over question-relevant KG

facts that traverse KG entities and lead to the

question answers. To facilitate reasoning, the

question is decoded into instructions, which

are dense question representations used to

guide the KG traversals. However, if the

derived instructions do not exactly match the

underlying KG information, they may lead

to reasoning under irrelevant context. Our

method, termed REAREV, introduces a new

way to KGQA reasoning with respect to

both instruction decoding and execution. To

improve instruction decoding, we perform

reasoning in an adaptive manner, where KG-

aware information is used to iteratively update

the initial instructions. To improve instruction

execution, we emulate breadth-ﬁrst search

(BFS) with graph neural networks (GNNs).

The BFS strategy treats the instructions as a set

and allows our method to decide on their exe-

cution order on the ﬂy. Experimental results

on three KGQA benchmarks demonstrate the

REAREV’s effectiveness compared with previ-

ous state-of-the-art, especially when the KG is

incomplete or when we tackle complex ques-

tions. Our code is publicly available at https:

//github.com/cmavro/ReaRev_KGQA.

1 Introduction

A knowledge graph (KG) is a relational graph that

contains a set of known facts. These facts are rep-

resented as tuples (subject, relation, object), where

the subject and object are KG entities that link with

the given relation. The knowledge graph question-

answering (KGQA) task takes as input a natural

language question and returns a set of KG entities

as the answer. In the weakly-supervised KGQA

setting (Berant et al.,2013), the only available

supervisions during learning are question-answer

Pulp Fiction

Q. Tarantino

Kill Bill

Q: “Which year are Q.Tarantino’s movies directed?”

A: 2003, 1994

KG links2003

direct

1963

date

1994

cast

aired on

act

U.Thurman

correct KG traversals

instruction

decoding

i(1), i(2)

direct

Execution i(1) −→ i(2) 7: Tarantino date

−−→ 7

Execution i(2) −→ i(1) 3: Tarantino direct

−−−→Kill Bill date

−−→2003 3

Decoding 7: Tarantino direct

−−−→ aired on

−−−−→ 7

Figure 1: The given question is decomposed into in-

structions {i(1), i(2)}that are matched with relations

date and direct, respectively. If the instructions are ex-

ecuted in an incorrect order (i(1) −→ i(2)), we cannot

arrive at the answer. Moreover, if the instructions can-

not be matched with the relation aired on (right KG

traversal), we cannot arrive at the second answer.

pairs, e.g., “Q: Who is the director of Pulp Fic-

tion? A: Q. Tarantino”. Due to labelling costs,

the ground-truth KG traversals, such as the path

Pulp Fiction director

−−−−→ Q. Tarantino

, whose execu-

tion leads to the answers, are seldom available.

The KGQA problem involves two modules: (i)

retrieving question-relevant KG facts, and (ii) rea-

soning over them to arrive at the answers. For

the reasoning module, a general approach (Lan

et al.,2021) is to decode the question into dense

representations (instructions) that guide the rea-

soning over the KG. The instructions are matched

with one-hop KG facts (or relations) in an iter-

ative manner, which induces KG traversals that

lead to the answers. Instructions are usually gener-

ated by attending to different question’s parts, e.g.,

“...is the director...” (Qiu et al.,2020a). KG traver-

sals are typically performed by utilizing powerful

graph reasoners, such as graph neural networks

(GNNs) (Kipf and Welling,2016;Sun et al.,2018).

The main challenge is that question answering is

performed over rich graph structures with possibly

arXiv:2210.13650v1 [cs.CL] 24 Oct 2022

complex semantics, and thus, instruction decoding

and execution is a key factor for KGQA. Figure 1

shows an example where suboptimal initial instruc-

tions lead to incorrect KG traversals. While some

methods (Miller et al.,2016;Zhou et al.,2018;Sun

et al.,2018;Xu et al.,2019) attempt to improve the

quality of the instructions, they are mainly designed

to tackle speciﬁc question types, such as 2-hop or

3-hop questions, or show poor performance for

complex questions (Sun et al.,2019).

Our method termed REAREV (Reason & Re-

vise) introduces a new way to KGQA reasoning

with respect to both instruction execution and de-

coding. To improve instruction execution, we do

not use instructions in a pre-deﬁned (possibly in-

correct) order, but allow our method to decide on

the execution order on the ﬂy. We achieve this by

emulating breadth-ﬁrst search (BFS) with GNN

reasoners. The BFS strategy treats the instructions

as a set and the GNN decides which instructions to

accept. To improve instruction decoding, we reason

over the KG to obtain KG-aware information and

use this information to adapt the initial instructions.

Then, we restart the reasoning with the new instruc-

tions that are conditioned to the underlying KG

semantics. To the best of our knowledge, adaptive

reasoning with emulating graph search algorithms

with GNNs has not been previously proposed for

KGQA.

We empirically show that REAREV performs ef-

fective reasoning over KGs and outperforms other

state-of-the-art. For KGQA with complex ques-

tions, REAREV achieves improvement for 4.1 per-

centage points at Hits@1 over the best competing

approach.

Our contributions are summarized below:

•

We improve instruction decoding via adaptive

reasoning, which updates the instructions with

KG-aware information.

•

We improve instruction execution by emu-

lating the breadth-ﬁrst search algorithm with

graph neural networks, which provides robust-

ness to the instruction ordering.

•

We achieve state-of-the-art (or nearly) perfor-

mance on three widely used KGQA datasets:

WebQuestions (Yih et al.,2015), Complex We-

bQuestions (Talmor and Berant,2018), and

MetaQA (Zhang et al.,2018).

2 Related Work

There are two mainstream approaches to solve

KGQA: (i) parsing the question to executable KG

queries like SPARQL, and (ii) grounding question

and KG representations to a common space for

reasoning.

Regarding the ﬁrst case, early methods (Berant

et al.,2013;Reddy et al.,2014;Bast and Hauss-

mann,2015) rely on pre-deﬁned question templates

to synthesize queries, which requires strong do-

main knowledge. Recent methods (Yih et al.,2015;

Abujabal et al.,2017;Luo et al.,2018;Bhutani

et al.,2019;Lan et al.,2019;Lan and Jiang,2020;

Qiu et al.,2020b;Sun et al.,2021;Das et al.,2021)

use deep learning techniques to automatically gen-

erate such executable queries. However, they need

ground-truth executable queries as supervisions

(which are costly to obtain) and their performance

is limited when the KG has missing links (non-

executable queries).

Methods in the second category alleviate the

need for ground-truth queries by learning natural

language and KG representations to reason in a

common space. These methods match question

representations to KG facts (Miller et al.,2016;Xu

et al.,2019;Atzeni et al.,2021) or to KG struc-

ture representations (Zhang et al.,2018;Han et al.,

2021;Qiu et al.,2020a). More related to our work,

GraftNet (Sun et al.,2018), PullNet (Sun et al.,

2019), and NSM (He et al.,2021) enhance the rea-

soning with graph-based reasoners. Our approach

aims at improving the graph-based reasoning via

adaptive instruction decoding and execution.

Researchers have also considered the problem of

performing KGQA over incomplete graphs (Min

et al.,2013), where important information is miss-

ing. These methods either rely on KG embed-

dings (Saxena et al.,2020;Ren et al.,2021) or

on side information, such as text corpus (Sun et al.,

2018,2019;Xiong et al.,2019;Han et al.,2020),

to infer missing information. However, they offer

marginal improvement over other methods when

the KG is mostly complete.

KGQA has also been adapted to speciﬁc do-

mains, such as QA over temporal (Mavromatis

et al.,2021;Saxena et al.,2021) and common-

sense (Speer et al.,2017;Talmor et al.,2019;Lin

et al.,2019;Feng et al.,2020;Yasunaga et al.,

2021) KGs.

3 Background

3.1 KG

A KG

G:= (V,R,F)

contains a set of entities

a set of relations

, and a set of facts

. Each fact

(v, r, v0)∈ F

is a tuple where

v, v0∈ V

denote the

subject and object entities, respectively, and

r∈ R

denotes the relation that holds between them. A

KG is represented as a directed graph with

|R|

relation types, where nodes

and

connect with

a directed relation

(v, r, v0)∈ F

. Nodes and

relations are usually initialized with

-dimensional

vectors (representations).

We denote

hv∈Rd

and

r∈Rd

the repre-

sentations for node

and relation

, respectively.

We denote

Ne(v)

and

Nr(v)

the set of node

’s

neighboring entities and relations (including self-

links), respectively. For example,

v0∈ Ne(v)

and

r∈ Nr(v)

if a fact

(v0, r, v)

exists. We use the

terms nodes and entities interchangeably.

3.2 KGQA

Given a KG

G:= (V,R,F)

and a natural language

question

, the task of KGQA is to extract a set of

entities

{a} ∈ V

that correctly answer

. Follow-

ing the standard setting in KGQA (Sun et al.,2018),

we assume that the entities referred in the question

are given and linked to nodes of

via entity linking

algorithms (Yih et al.,2015). We denote these enti-

ties as

{e}q∈ V

(seed entities), e.g., Q. Tarantino

in Figure 1.

The problem complexity is further reduced by

extracting a question-speciﬁc subgraph

Gq:=

(Vq,Rq,Fq)⊂ G

which is likely to contain the

answers (more details in Section 5). Each ques-

tion

and its answers

{a}q∈ Vq

, referred to as

question-answer pair, induces a question-speciﬁc

labeling of the nodes. Node

v∈ Gq

has label

yv= 1

v∈ {a}q

and

yv= 0

otherwise. The

task can be thus reduced to performing binary node

classiﬁcation over Gq.

The KGQA problem involves two modules: (i)

retrieving a question-speciﬁc

and (ii) reasoning

over

to perform answer classiﬁcation. In this

work, we introduce a new way to advance KGQA

reasoning capabilities.

3.3 GNNs

GNNs (Kipf and Welling,2016;Schlichtkrull et al.,

2018) are well-established graph representation

learners suited for tasks such as node classiﬁcation.

Following the message passing strategy (Gilmer

et al.), the core idea of GNNs is to update the rep-

resentation of each node by aggregating itself and

its neighbors’ representations.

The GNN updates node representation

h(l)

layer las

h(l)

v=ψh(l−1)

v, φ{m(l)

v0v:v0∈ Ne(v),(1)

where

m(l)

v0v

is the message between two neigh-

bors

and

, and

φ(·)

is an aggregation function

of all neighboring messages. Function

ψ(·)

com-

bines representations of consecutive layers. At each

layer, GNNs capture 1-hop information (neighbor-

ing messages). An

-layer GNN model captures

the neighborhood structure and semantics within

hops.

3.4 GNNs for KGQA

To better reason over multiple facts (graphs), suc-

cessful KGQA methods utilize GNNs (Sun et al.,

2018;He et al.,2021). The idea is to condition the

message passing of Eq.(1) to the given question

For example, if a question refers to movies, then

1-hop movie entities are more important. It is com-

mon practice (Qiu et al.,2020a;He et al.,2021;Shi

et al.,2021;Lan et al.,2021) to decompose

into

representations

{i(l)}L

l=1

(instructions), where

each one may refer to a speciﬁc question’s context,

e.g., movies or actors.

The instructions are used to guide different rea-

soning steps over

by writing the GNN updates

h(l)

v=ψh(l−1)

v, φ{m(l)

v0v:v0∈ Ne(v)|i(l)},

(2)

where each GNN layer

is now conditioned to a

different instruction

i(l)

. Message

m(l)

v0v

usually de-

pends on the representations of the corresponding

fact (v0, r, v).

The goal of GNNs is to selectively aggregate

information from the question-relevant facts. Via

Eq.(2), GNNs learn to match each

i(l)

with 1-hop

neighboring facts. Using the instructions

{i(l)}L

l=1

recursively, GNNs learn the sequence of facts (KG

traversal) that leads to the ﬁnal answers.

4 REAREV Approach

REAREV (Reason & Revise) enhances instruction

decoding and execution for effective KGQA reason-

ing. Our contributions across these two dimensions

are described below.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ReaRev:AdaptiveReasoningforQuestionAnsweringoverKnowledgeGraphsCostasMavromatisandGeorgeKarypisUniversityofMinnesota,USA{mavro016,karypis}@umn.eduAbstractKnowledgeGraphQuestionAnswering(KGQA)involvesretrievingentitiesasan-swersfromaKnowledgeGraph(KG)usingnaturallanguagequeries.Thechallengeistolearnt...

展开>> 收起<<

ReaRev Adaptive Reasoning for Question Answering over Knowledge Graphs Costas Mavromatis andGeorge Karypis.pdf

共12页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

ReaRev Adaptive Reasoning for Question Answering over Knowledge Graphs Costas Mavromatis andGeorge Karypis

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: