TwiRGCN Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs Aditya Sharma

2025-04-26 0 0 1.23MB 12 页 10玖币

侵权投诉

TwiRGCN: Temporally Weighted Graph Convolution for Question

Answering over Temporal Knowledge Graphs

Aditya Sharma

IISc, Bangalore

adityasharma@iisc.ac.in

Apoorv Saxena

IISc, Bangalore

apoorvsaxena@iisc.ac.in

Chitrank Gupta

IIT Bombay

chigupta2011@gmail.com

Mehran Kazemi

Google Research, Montreal

mehrankazemi@google.com

Partha Talukdar

Google Research, India

partha@google.com

Soumen Chakrabarti

IIT Bombay

soumen@cse.iitb.ac.in

Abstract

Recent years have witnessed interest in Tem-

poral Question Answering over Knowledge

Graphs (TKGQA), resulting in the develop-

ment of multiple methods. However, these

are highly engineered, thereby limiting their

generalizability, and they do not automatically

discover relevant parts of the KG during multi-

hop reasoning. Relational graph convolutional

networks (RGCN) provide an opportunity to

address both of these challenges – we explore

this direction in the paper. Speciﬁcally, we pro-

pose a novel, intuitive and interpretable scheme

to modulate the messages passed through a KG

edge during convolution based on the relevance

of its associated period to the question. We also

introduce a gating device to predict if the an-

swer to a complex temporal question is likely to

be a KG entity or time and use this prediction to

guide our scoring mechanism. We evaluate the

resulting system, which we call TwiRGCN, on

a recent challenging dataset for multi-hop com-

plex temporal QA called TimeQuestions. We

show that TwiRGCN signiﬁcantly outperforms

state-of-the-art models on this dataset across di-

verse question types. Interestingly, TwiRGCN

improves accuracy by 9–10 percentage points

for the most difﬁcult ordinal and implicit ques-

tion types.

1 Introduction

Question answering (QA) is a key problem in nat-

ural language processing and a long-lasting mile-

stone for artiﬁcial intelligence. A large class of

approaches for QA makes use of knowledge graphs

(KG), which are multi-relational graphs represent-

ing facts (KGQA). Temporal KGs (TKG) represent

facts that are only valid for speciﬁc periods of time

as (subject, relation, object, time range), for ex-

ample, (Franklin D Roosevelt, position held, Pres-

ident of USA,

[1933,1945]

). The problem of an-

swering questions that require temporal reasoning

over TKGs (TKGQA) is a special case of KGQA

that speciﬁcally focuses on the following challenge:

temporal questions constrain answers through tem-

poral notions, e.g., “who was the ﬁrst president of

US during WW2?” Developing systems for tem-

poral QA is of immense practical importance for

many applications. It is considered a more chal-

lenging problem than KGQA (Bhutani et al.,2019;

Saxena et al.,2020), where questions are typically

about persistent, non-temporal facts (e.g., place of

birth), with only a small portion of the questions

requiring any temporal reasoning (Jia et al.,2018a).

Even though a variety of models have been pro-

posed for the TKGQA recently, they suffer from the

following problems: 1) they are either highly engi-

neered toward the task (Jia et al.,2021;Chen et al.,

2022) or 2) they do not incorporate graph structure

information using Graph Neural Networks (GNN)

(Mavromatis et al.,2021;Shang et al.,2022;Sax-

ena et al.,2021). We explore the following hy-

potheses in this paper: 1) a simple GNN-based

solution could generalize better and offer higher

performance than highly engineered GNN-based,

and TKG embedding-based models; 2) a multi-

layer GNN model could do multi-hop reasoning

across its layers; 3) not all edges (temporal facts)

are equally important for answering temporal ques-

tions (see Figure 1), so GNN solutions could beneﬁt

from temporally weighted edge convolutions.

Following the aforementioned hypotheses, we

develop a novel but architecturally simple TKGQA

system that we call “Temporally weighted Rela-

tional Graph Convolutional Network” (TwiRGCN).

It is based on the Relational Graph Convolu-

tional Network (RGCN) proposed by Schlichtkrull

et al. (2018). TwiRGCN introduces a question-

dependent edge weighting scheme that modulates

convolutional messages passing through a temporal

fact edge based on how relevant the time period

of that edge is for answering a particular ques-

tion. In RGCN, convolution messages from all

TKG edges are weighted equally. But all edges

are not equally important for answering temporal

arXiv:2210.06281v2 [cs.CL] 6 Oct 2023

questions. For example, in Figure 1, to answer

the question “Who was the ﬁrst president of the

US during WW2?” the edge with Bill Clinton has

little relevance for answering the question. But,

regular RGCN would still weigh all edges equally.

We address this shortcoming through our proposed

modulation. We impose soft temporal constraints

on the messages passed during convolution, am-

plifying messages through edges close to the time

period relevant for answering the question while

diminishing messages from irrelevant edges. This

leads to better, more efﬁcient learning as we are

not confusing our model with unnecessary infor-

mation, as evidenced by our signiﬁcantly improved

performance without the need for any heavy engi-

neering. We explore two different strategies for our

convolutional edge weighting, which show comple-

mentary strengths. Our experiments establish that

TwiRGCN signiﬁcantly outperforms already strong

baselines on TimeQuestions. Our contributions are:

•

We propose TwiRGCN, a simple and gen-

eral TKGQA system that computes question-

dependent edge weights to modulate RGCN mes-

sages, depending on the temporal relevance of

the edge to the question.

•

We explore two novel and intuitive schemes for

imposing soft temporal constraints on the mes-

sages passed during convolution, amplifying mes-

sages through edges close to the time relevant for

answering the question while diminishing mes-

sages from irrelevant edges. We also propose

an answer-gating mechanism based on the likeli-

hood that the answer is an entity or time.

•

Through extensive experiments on a challeng-

ing real-world dataset, we ﬁnd that TwiRGCN

substantially outperforms prior art in overall ac-

curacy, and by 9–10% on the implicit and ordinal

type questions — categories that require signiﬁ-

cant temporal reasoning.

•

We augment TimeQuestions with a TKG and re-

lease both code and data at https://github.com/adi-

sharma/TwiRGCN.

2 Related Work

Most KGQA systems have focused on answering

questions from simple (i.e., 1-hop fact-based ques-

tions) (Berant et al.,2013) to multi-hop complex

questions requiring multi-fact reasoning (Sun et al.,

2019;Saxena et al.,2020). However, only a small

fraction of these questions require any temporal

reasoning (Jia et al.,2018a). Recent efforts have

President of the

United States

position held

[1913, 1921]

position held

[1993, 2001]

position held

[1945, 1953]

Franklin D.

Roosevelt

position held

[1933, 1945]

Bill Clinton

Harry

Truman

Woodrow

Wilson

Who was the first president of US during WW2?

where

Figure 1: An illustrative example of how our temporal gating

described in Section 4.2 modulated the incoming graph con-

volution messages for one node depending on the time period

of interest for the question. The thickness of an edge here is

proportional to the value of the temporal edge weight

m(e)

for that edge. In this example, the entities Franklin D. Roo-

sevelt and Harry Truman, who were presidents during WW2

[1939,1945]

get the top two highest weights, while Woodrow

Wilson, who was president during WW1

[1914,1918]

gets a

smaller edge weight. In contrast, Bill Clinton, whose time

period is unrelated to the question, gets a much lower edge

weight. Thus, contributing very little to the convolution update

of the ’President of the US’ node.

tried to overcome this gap by proposing models

as well as datasets to explicitly focus on temporal

reasoning. We review these below.

Temporal KGQA methods: One line of work

uses temporal constraints along with hand-crafted

rules to ﬁnd the answer (Bao et al.,2016;Luo

et al.,2018;Jia et al.,2018b). A recent class of

models has leveraged advances in TKG embed-

ding methods for answering questions on Tempo-

ral KGs. CronKGQA (Saxena et al.,2021) does

this by posing a question as a TKG completion

problem and ﬁnds the answer using the TComplex

(Lacroix et al.,2020) score function and BERT (De-

vlin et al.,2018) question embedding to complete

the fact. TempoQR (Mavromatis et al.,2021) uses

additional temporal supervision to enrich TKG em-

beddings, followed by a transformer-based decoder

(Vaswani et al.,2017). TSQA (Shang et al.,2022)

on the other hand estimate the time in the question

and uses it to enrich TKG embeddings for ﬁnding

the answer. SubGTR (Chen et al.,2022) infers

question-relevant temporal constraints using TKG

embeddings and applies them as ﬁlters to score en-

tities in the question subgraph. Although we, too,

use pre-trained TKG embeddings to initialize our

generalized RGCN, we use the GNN framework to

take advantage of the structural information in the

KG in ways that they do not. Recent work (Teru

et al.,2020) shows that GNN-based models can

encode any logical rule corresponding to a path in

the knowledge graph. We refer to this as structural

information that shallow embedding-based models

cannot access.

RGCN based QA systems: Graph neural networks

are increasingly being used in QA systems not

speciﬁcally meant for temporal reasoning. Graft-

Net (Sun et al.,2018) uses personalized PageRank

to collect a query-relevant subgraph from a global

KG, then an RGCN to predict the answer from

the relevant subgraph. PullNet (Sun et al.,2019)

loops over and expands GraftNet’s subgraph to do

multi-hop reasoning. EXAQT (Jia et al.,2021) is

the system closest to ours: it addresses TKGQA

and also uses an RGCN. The RGCN for answer

prediction which works on the question subgraph

is very similar to that in GraftNet. EXAQT aug-

ments it with dictionary matching, heavy engineer-

ing, and additional category information. In con-

trast, TwiRGCN uses a straightforward temporally

weighted graph convolution followed by answer

gating, as described in Section 4, while still achiev-

ing superior performance (see Section 5.3). More

details in Section 5.2.

3 Preliminaries

3.1 Temporal Knowledge Graphs (TKG)

KG: Multi-relational graphs with entities (eg:

Barack Obama, USA) as nodes and relations

between entities

{s, o}

(e.g., president of) rep-

resented as typed edges between nodes. Each

edge of this graph, together with endpoint

nodes, represents a fact triple

{s, r, o}

, e.g.,

{Barack Obama,president of,USA}.

TKG: Numerous facts in the world are not perpetu-

ally true and are only valid for a certain time period.

A TKG represents such a fact as a quadruple of the

form

{s, r, o, [tst, tet]}

, where

tst

is the start time

and

tet

is the end time of validity of the fact, e.g.,

{Barack Obama,president of,USA,[2009,2017]}

3.2 Question Answering on TKGs

Given a question

speciﬁed in natural language

form and a TKG

, TKGQA is the task of ﬁnd-

ing the answer to

based on the information that

is available (or can be derived) from

. A sub-

graph of

is a subset of its nodes with induced

edges. In this paper, we assume each question is al-

ready associated with a subgraph

relevant to the

question. We deﬁne

Gq= (Vq,Rq,Tq,Eq)

as the

subgraph of

associated with a question

q∈ Q

where

represents the set of all questions. Each

edge

e∈ Eq

represents a fact

{vi, r, vj,[tst, tet]}

where

vi, vj∈ Vq

are entity nodes,

r∈ Rq

is the

relation between them and

tst, tet ∈ Tq

are the start

and end times for which the fact is valid.

3.3

Relational Graph Convolutional Networks

Given a KG, each node

is initialized to a suitable

embedding

h(0)

at layer 0.Thereafter, Schlichtkrull

et al. (2018) propose to update node embeddings

h(l+1)

vi, at layer (l+ 1), as follows:

h(l+1)

vi=σ

X

r∈R X

j∈N r

W(l)

rh(l)

|N r

i|+W(l)

0h(l)

vi

(1)

where

is the set of neighbors of node

that are

connected via relation edges of type

is the set

of relations,

W(l)

are weight matrices associated

with each relation type

and layer

. They are

initialized using a basis decomposition method.

4 Proposed Method: TwiRGCN

In this section, we develop and describe TwiRGCN

(“Temporally Weighted Relational Graph Convolu-

tional Network”), our model for TKGQA.

4.1 Embedding for questions and KG facts

Question embedding: We pass the question text

through a pre-trained encoder-only language model

(LM) to obtain a question embedding. In particular,

we prepend a [CLS] token to the input question

and feed it into BERT (Devlin et al.,2019), and

then use its output-layer [CLS] embedding as the

question embedding

. We enable LM ﬁne-tuning

during training.

TKG preprocessing for RGCN initialization: We

initialize entity and time embeddings using pre-

trained TComplEx (Lacroix et al.,2020) embed-

dings.

To obtain these for the TimeQuestions

dataset (Jia et al.,2021), we ﬁrst construct a ‘back-

ground KG’

G=Sq∈Q Gq

which is the union of

all question subgraphs

in the train dataset. As

in most temporal KGQA works, we discretize time

to a suitable granularity (in our dataset, a year).

The graph on which TwiRGCN is run represents

every entity as a node

and time as edge attribute

. Their initial (layer-0) RGCN embeddings

h(0)

and

htj

, are set to the entity and time embeddings

TComplEx is known to provide high-quality embeddings,

but other TKG embedding methods such as TimePlex (Jain

et al.,2020) can also be used.

TwiRGCN can be extended to TKGQA datasets that do

not provide subgraphs through recently proposed subgraph

selection methods (Chen et al.,2022;Shang et al.,2022).

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

TwiRGCN:TemporallyWeightedGraphConvolutionforQuestionAnsweringoverTemporalKnowledgeGraphsAdityaSharmaIISc,Bangaloreadityasharma@iisc.ac.inApoorvSaxenaIISc,Bangaloreapoorvsaxena@iisc.ac.inChitrankGuptaIITBombaychigupta2011@gmail.comMehranKazemiGoogleResearch,Montrealmehrankazemi@google.comParthaTaluk...

展开>> 收起<<

TwiRGCN Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs Aditya Sharma.pdf

共12页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

TwiRGCN Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs Aditya Sharma

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: