Mining Causality from Continuous-time Dynamics Models An Application to Tsunami Forecasting Fan WuSanghyun HongDonsub RimNoseong ParkKookjin Lee

2025-05-02 0 0 1.13MB 10 页 10玖币

侵权投诉

Mining Causality from Continuous-time Dynamics Models:

An Application to Tsunami Forecasting

Fan Wu∗Sanghyun Hong†Donsub Rim‡Noseong Park§Kookjin Lee¶

Abstract

Continuous-time dynamics models, such as neural ordinary

diﬀerential equations, have enabled the modeling of under-

lying dynamics in time-series data and accurate forecast-

ing. However, parameterization of dynamics using a neu-

ral network makes it diﬃcult for humans to identify causal

structures in the data. In consequence, this opaqueness hin-

ders the use of these models in the domains where capturing

causal relationships carries the same importance as accurate

predictions, e.g., tsunami forecasting. In this paper, we ad-

dress this challenge by proposing a mechanism for mining

causal structures from continuous-time models. We train

models to capture the causal structure by enforcing sparsity

in the weights of the input layers of the dynamics models.

We ﬁrst verify the eﬀectiveness of our method in the sce-

nario where the exact causal-structures of time-series are

known as a priori. We next apply our method to a real-

world problem, namely tsunami forecasting, where the exact

causal-structures are diﬃcult to characterize. Experimental

results show that the proposed method is eﬀective in learn-

ing physically-consistent causal relationships while achieving

high forecasting accuracy.

Keywords: Causality mining, Neural ordinary diﬀerential

equations, Tsunami forecasting

1 Introduction.

An emerging paradigm for modeling the underlying dy-

namics in time-series data is to use continuous-time

dynamics models, such as neural ordinary diﬀeren-

tial equations (NODEs) [2, 28, 5]. Widely known as

a continuous-depth extension of residual networks [8],

NODEs have a great ﬁt for data-driven dynamics mod-

eling as they construct models in the form of systems of

ODEs. This new paradigm has enabled breakthroughs

in many applications, such as patient status/human

activity prediction [28], computational physics prob-

lems [12, 13, 14], or climate modeling [11, 24].

A promising approach to improve the performance

of those models is to increase the expressivity of the

neural network they use to parameterize a system,

e.g., by employing convolutional neural networks or

∗Arizona State University, fanwu8@asu.edu

†Oregon State University, sanghyun.hong@oregonstate.edu

‡Washington University in St. Louis, rim@wustl.edu

§Yonsei University, noseong@yonsei.ac.kr

¶Arizona State University, kookjin.lee@asu.edu

by augmenting extra dimension in the state space [5].

However, the more complex neural networks are, the

more challenging humans are to interpret the learned

models. It will be even more problematic for tasks that

require interpretable causality,e.g., a high-consequence

event prediction, as in tsunami forecasting.

In this work, we study a mechanism for mining

the causality in time-series data from continuous-time

neural networks trained on it. Speciﬁcally, we ask:

How can we identify causal structures from a

continuous-time model? How can we promote

the model to learn causality in time-series?

This work focuses on Granger causality [6], a common

framework that are used to quantify the impact of a

past event observed on the future evolution of data.

Our contributions. First, we propose a mechanism

for extracting the causal structures from the parameters

of a continuous-time neural network. We adapt the con-

cept of component-wise neural networks, studied in [33],

for continuous-time models. We enforce column-wise

sparsity to the weights of the input layers so that the

impact of the input elements of less contributions (less

causal) can be small. This is achieved by using a train-

ing algorithm, which minimizes data-matching loss and

sparsity-promoting loss and prunes the columns whose

weights are smaller than a threshold. At the end of the

training, the norms of columns corresponding to the

important input elements will have high magnitudes,

which will make interpretation on causality easier.

Second, we test our approach if the learned causal

structures match with the known ground-truth causal

structures. To evaluate, we train two continuous-time

dynamics models, NODEs [2] and neural delay diﬀeren-

tial equations (NDDEs) [37], on the data sampled from

the Lorenz-96 and Mackey–Grass systems, respectively.

The results demonstrate that our mechanism precisely

extracts the causal structures from the data.

Third, we further evaluate our approach in tsunami

forecasting at the Strait of Juan de Fuca [22], where

we do not know the exact causal relationships in the

data. We train NDDEs on the tsunami dataset [20]

arXiv:2210.04958v2 [cs.LG] 13 Oct 2022

by using the proposed training algorithm. We achieve

comparable accuracy with the prior work [17] and

eﬀective in predicting the highest peaks of sea-surface

elevations at locations near highly-populated areas.

Furthermore, we show that one can approximately

capture the physically-consistent causal relationships

between a tsunami event and tide observed before,

which agrees with the domain expert’s interpretation.

This result suggests that we can accurately model

the underlying dynamics in time-series data using

continuous-time dynamics models while capturing the

causality. Future work will explore adopting diverse

causality deﬁnitions and apply to various applications.

2 Preliminaries

2.1 Neural Ordinary Diﬀerential Equations

Neural ordinary diﬀerential equations (NODEs) [2] are

a family of deep neural networks that parameterize the

time-continuous dynamics of hidden states in the data

as a system of ODEs using a neural network:

(2.1) dz

dt=fΘ(z, t),

where z(t)∈Rnzdenotes a time-continuous hidden

state, fΘdenotes a velocity function, parameterized

by a neural network whose parameters are denoted as

Θ. A typical parameterization of fΘis a multi-layer

perceptron (MLP) Θ={(W`,b`)}`=1.W`and b`are

weights and biases of the `-th layer, respectively.

The forward pass of NODEs is equivalent to solving

an initial value problem via a black-box ODE solver.

Given an initial condition z0and fΘ, it computes:

z(t0),z(t1) = ODESolve(z0,fΘ,{t0, t1}).

2.2 Neural Delay Diﬀerential Equations

Although NODEs serve as a great means for data-driven

dynamics modeling, they exhibit several limitations.

One obvious limitation in the context of causality mod-

eling is that NODEs take only the current state of the

input variables z(t) as shown in Eq. (2.1) and, thus, are

not suitable for capturing delayed causal eﬀects.

To resolve this issue, we employ the computational

formalism provided by neural delay diﬀerential equa-

tions (NDDEs) [37], an extension of NODEs, which

takes extra input variables z≤τ(t) as follows:

dt=fΘ(z(t),z≤τ(t), t),

where z≤τ(t) = {z(t−γ) : γ∈[0, τ]}denotes the

trajectory of the solution in the past up to time t−τ.

To avoid numerical challenges in handling a continuous

form of delay, we choose to consider a discrete form of

delay:

(2.2) dz

dt=fΘ(z(t),z(t−τ1),...,z(t−τm), t).

2.3 Neural Granger Causality

Granger causality [6] has been a common choice for dis-

covering a structure that quantiﬁes the extent to which

one time series aﬀects predicting the future evolution of

another time series. When the linear dynamics model is

considered, such as a vector autoregressive model, where

the dynamics are simply represented as a linear trans-

formation, e.g.,zt=Azt−1. Granger causality can be

identiﬁed by the structure of the matrix A. If the i-th

column of j-th row is zero, it can be interpreted that

the i-th series does not Granger cause the j-th series.

For time series that evolves according to general nonlin-

ear dynamics, however, it is often challenging to embed

such structure in a nonlinear dynamics model.

Recent work [33] proposed to adapt neural network

architectures in a way that the Granger causal interac-

tions (or neural Granger causality) are estimated while

performing data-driven dynamics modeling of nonlin-

ear systems. They presented component-wise MLPs or

component-wise recurrent neural networks, where the

individual components of the output are constructed as

separate neural networks and, thus, the eﬀects of in-

put on individual output series can be captured. To

promote the sparse causality structure for better inter-

pretability, their method imposes a penalty on the in-

put layers of each component neural networks. In this

work, we translate their causality learning frameworks

into NODEs and NDDEs.

3 Mining Causality in Time-series Data

We now deﬁne neural Granger causality in the context

of continuous-time dynamics models. We then propose

a mechanism for mining the causality in time-series via

training continuous-time dynamics models on it.

3.1 Neural Granger Causality in

Continuous-time Dynamics Models

As a multi-variate time-series, in continuous-time mod-

els, represented as an implicit form, the rate of changes

of the variables with respect to time t, we deﬁne neural

Granger causality in continuous-time models as follows:

Definition 3.1. Let us call the time-series zjwith

delay τkas the (j, k)-th delayed time-series zj,k(t) =

zj(t−τk). We say (j, k)-th delayed time series zj,k is

Granger non-causal in the system (2.2) with respect to

the i-th time-series zi, if ∀(z1,1, ..., zj,k, ..., znz,m)and

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MiningCausalityfromContinuous-timeDynamicsModels:AnApplicationtoTsunamiForecastingFanWu*SanghyunHongDonsubRimNoseongPark§KookjinLee¶AbstractContinuous-timedynamicsmodels,suchasneuralordinarydierentialequations,haveenabledthemodelingofunder-lyingdynamicsintime-seriesdataandaccurateforecast-ing.How...

展开>> 收起<<

Mining Causality from Continuous-time Dynamics Models An Application to Tsunami Forecasting Fan WuSanghyun HongDonsub RimNoseong ParkKookjin Lee.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Mining Causality from Continuous-time Dynamics Models An Application to Tsunami Forecasting Fan WuSanghyun HongDonsub RimNoseong ParkKookjin Lee

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: