Teaching Neural Module Networks to Do Arithmetic Jiayi Chen Xiao-Yu Guo Yuan-Fang Li and Gholamreza Haffari Faculty of Information Technology Monash University Melbourne Australia

2025-05-06 0 0 881.34KB 9 页 10玖币

侵权投诉

Teaching Neural Module Networks to Do Arithmetic

Jiayi Chen , Xiao-Yu Guo , Yuan-Fang Li , and Gholamreza Haffari

Faculty of Information Technology, Monash University, Melbourne, Australia

jche0069@student.monash.edu

{xiaoyu.guo,yuanfang.li,gholamreza.haffari}@monash.edu

Abstract

Answering complex questions that require

multi-step multi-type reasoning over raw text

is challenging, especially when conducting nu-

merical reasoning. Neural Module Networks

(NMNs), follow the programmer-interpreter

framework and design trainable modules to

learn different reasoning skills. However,

NMNs only have limited reasoning abilities,

and lack numerical reasoning capability. We

upgrade NMNs by: (a) bridging the gap

between its interpreter and the complex

questions; (b) introducing addition and

subtraction modules that perform numerical

reasoning over numbers. On a subset of

DROP, experimental results show that our

proposed methods enhance NMNs’ numerical

reasoning skills by 17.7% improvement of F1

score and signiﬁcantly outperform previous

state-of-the-art models.

1 Introduction

Complex Question Answering (CQA) over text is

a challenging task in Natural Language Understand-

ing (NLU). Based on the programmer-interpreter

paradigm, Neural Module Networks (NMNs)

(Gupta et al.,2020) learn to ﬁrst parse complex

questions as executable programs composed of

various predeﬁned trainable modules, and then

execute such programs (implemented by modules)

over the given paragraph to predict answers of

all kinds. NMNs achieve competitive reasoning

performance on a subset of DROP (Dua et al.,

2019), and possess remarkable interpretability that

is also important for CQA.

However, NMNs’ numerical reasoning capability

is insufﬁcient: it is incapable of handling arithmetic

operations such as addition and subtraction between

numbers, which make up nearly 40% questions of

the DROP dataset. Moreover, a gap exists between

the interpreter and the complex question since

there is no interaction between them. Motivated by

these, we propose two methods to improve NMNs’

numerical reasoning skills.

First, we incorporate the original question in the

interpreter, aiming to

directly provide question in-

formation in the “execution” process

, especially

number-related questions. The intuition behind is

that, in the original NMNs, questions participate

in the process only through the programmer. This

can cause a distance between queries and returns.

For example, in Figure 1, the ﬁrst row shows that

the original NMNs found the wrong event (i.e.,

‘besieged Sinj’) solely based on the paragraph

information. In contrast, our model NMNs

can

easily target the correct event (i.e., ‘Sinj ﬁnally

fell’) with the help of question information.

Second, we introduce new modules to

support

addition and subtraction of up to three num-

bers

. Endowing NMNs with the ability to support

arithmetic can greatly boost its overall performance

on DROP and beyond. For instance, in Figure

1, the second row shows that the original NMNs

improperly adopt the

find-num

module for the

addition question because the module set does not

cover such an arithmetic ability. To facilitate the

learning of the

add/sub

modules, we extract QA

pairs related to addition and subtraction from the

original DROP dataset to construct a new dataset

for training and evaluation.

Experimental results show that our methods

signiﬁcantly enhance NMNs’ numerical reasoning

capability. On a subset of DROP, our methods

improve F1 score by 17.7% absolute points, and

on ADD-SUB questions by 65.7% absolute points.

Compared to NumNet (Ran et al.,2019), which is

speciﬁcally designed for numerical reasoning, our

method outperforms it by 2.9% absolute F1 points.

2 Background and Related Work

Semantic Parsing

is a widely-adopted approach in

the compositional question answering (CQA) task,

which involves a number of reasoning steps. In this

arXiv:2210.02703v1 [cs.CL] 6 Oct 2022

Question type Paragraph Question Answers

date compare In the Morean War, the Republic of Venice besieged Sinj in October 1684 and then

again March and April 1685, but both times without success. … With the help of the

local population of Poljica as well as the Morlachs, the fortress of Sinj finally fell to the

Venetian army on 30 September 1686. On 1 September 1687 the siege of Herceg Novi

started, and ended with a Venetian victory on 30 September. …

Which happened first, the fell

of Sinj or the siege of Herceg

Novi?

Original NMNs: besieged

Sinj

±: Sinj finally fell

Ground-truth: the fell of Sinj

/ Sinj finally fell

Original NMNs’ program: (span(compare-date-lt(find find))) ±: (span(compare-date-lt(find find)))

addition /

subtraction

(2 numbers)

… In the first quarter, the Niners struck first as kicker Joe Nedney got a 47-yard field

goal. In the second quarter, the Saints took the lead with QB Drew Brees completing a

5-yard and a 33-yard TD pass to WR Lance Moore. San Francisco would answer with

Nedney's 49-yard field goal, yet New Orleans replied with Brees' 47-yard TD pass to

WR Robert Meachem. …

How many yards was Nedney's

combined field goal yards in the

first and second quarters?

Original NMNs: 47

±: 96

Ground-truth: 96

Original NMNs’ program: (find-num (filter (find)) ±: (addition(find-num(find))(find-num(find)))

addition /

subtraction

(3 numbers)

The Greek census 2011 recorded 9,903,268 Greek citizens (91.56%), 480,824 Albanian

citizens (4.44%), 75,915 Bulgarian citizens (0.7%), 46,523 Romanian citizenship

(0.43%), 34,177 Pakistani citizens (0.32%), 27,400 Georgia (country) citizens (0.25%)

and 247,090 people had other or unidentified citizenship (2.3%). …

How many more people were

Greek citizens compared to

Albanian and Bulgarian citizens

combined?

Original NMNs: 9903268

±: 9346529

Ground-truth: 9346529

Original NMNs’ program: (find-num (find-max-num (find))) ±: (subtraction (find-num(find)) (addition(find-num(find))(find-num(find))))

Figure 1: Three examples in the DROP dataset and the predictions by original NMNs and our improved model

NMNs±. The relevant tokens and their corresponding modules are highlighted.

approach, a programmer maps natural-language

questions into machine-readable representations

(logical forms), which are executed by an interpreter

to yield the ﬁnal answer. For instance, WNSMN

(Saha et al.,2021) uses a generalized framework of

dependency parsing inspired by the Stanford depen-

dency parse tree (Chen and Manning,2014) to parse

queries into noisy heuristic programs. Neural Mod-

ule Networks (Gupta et al.,2020) extend semantic

parsing by making interpreter a learnable function

with speciﬁed modules and executing the logical

forms from the programmer in a step-wise manner.

Neural Module Networks

initially is proposed to

overcome the Visual Question Answering (VQA)

problem (Andreas et al.,2016), where questions are

often compositional. Gupta et al. (2020) employs

the programmer-interpreter framework with atten-

tion (Vaswani et al.,2017) to tackle the CQA task.

Speciﬁcally, the programmer parses each question

into an executable program. The interpreter takes

the program as input and perform various symbolic

reasoning functions. The modules are deﬁned in

a differentiable way, aiming to maintain the uncer-

tainty about each intermediate decision output and

propagate them through layers. For instance, the

predicted program of the ﬁrst example in Figure 1is

span(compare-date-lt(find,find))

The interpreter would ﬁrst calls the

find

module

twice to ﬁnd events queried by the question (e.g.,

‘the fell of Sinj’) and outputs appropriate paragraph

attention. The

compare-date-lt

module can

further locate the dates (e.g., ‘30 September 1686’)

to compute their relation. By demonstrating the

intermediate reasoning steps in this manner, NMNs

perform interpretable problem-solving.

Numerical Reasoning

is a necessary ability for

models to handle the CQA task (Geva et al.,

2020). Dua et al. (2019) modify the output

layer of QANet (Yu et al.,2018) and propose

a number-aware model NAQANet to deal with

numerical questions. NumNet (Ran et al.,2019)

leverage Graph Neural Network to capture relations

between numbers. Similarly, QDGAT (Chen et al.,

2020a) distinguish number types more precisely by

adding the connection with entities and obtained

better performance. Nerd (Chen et al.,2020b)

search possible programs exhaustively based on

answers and employed these programs as weak

supervision. Another similar work (Guo et al.,

2021) proposes a question-aware interpreter but

uses a totally different approach to measure the

alignment between the question and the context

paragraph. Though these approaches can achieve

the high performance on DROP dataset, it is

incomprehensible for the reasoning procedure.

3 Model

In this section, we tend to illustrate our proposed

methods. Basically, we will show the incorporation

of questions in Section 3.1. In Section 3.2, the

newly extended module: addition and subtraction

will be described.

3.1 The Incorporation of Questions

Taking one module

compare-date

as a case

study: it performs comparisons between two

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

TeachingNeuralModuleNetworkstoDoArithmeticJiayiChen,Xiao-YuGuo,Yuan-FangLi,andGholamrezaHaffariFacultyofInformationTechnology,MonashUniversity,Melbourne,Australiajche0069@student.monash.edu{xiaoyu.guo,yuanfang.li,gholamreza.haffari}@monash.eduAbstractAnsweringcomplexquestionsthatrequiremulti-stepmul...

展开>> 收起<<

Teaching Neural Module Networks to Do Arithmetic Jiayi Chen Xiao-Yu Guo Yuan-Fang Li and Gholamreza Haffari Faculty of Information Technology Monash University Melbourne Australia.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Teaching Neural Module Networks to Do Arithmetic Jiayi Chen Xiao-Yu Guo Yuan-Fang Li and Gholamreza Haffari Faculty of Information Technology Monash University Melbourne Australia

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: