MultiScale MeshGraphNets Meire Fortunato 1Tobias Pfaff 1Peter Wirnsberger1Alexander Pritzel1Peter Battaglia1 Abstract

2025-04-29 0 0 7.26MB 14 页 10玖币

侵权投诉

MultiScale MeshGraphNets

Meire Fortunato * 1 Tobias Pfaff * 1 Peter Wirnsberger 1Alexander Pritzel 1Peter Battaglia 1

Abstract

In recent years, there has been a growing inter-

est in using machine learning to overcome the

high cost of numerical simulation, with some

learned models achieving impressive speed-ups

over classical solvers whilst maintaining accu-

racy. However, these methods are usually tested

at low-resolution settings, and it remains to be

seen whether they can scale to the costly high-

resolution simulations that we ultimately want to

tackle.

In this work, we propose two complementary ap-

proaches to improve the framework from Mesh-

GraphNets, which demonstrated accurate predic-

tions in a broad range of physical systems. Mesh-

GraphNets relies on a message passing graph

neural network to propagate information, and

this structure becomes a limiting factor for high-

resolution simulations, as equally distant points in

space become further apart in graph space. First,

we demonstrate that it is possible to learn accu-

rate surrogate dynamics of a high-resolution sys-

tem on a much coarser mesh, both removing the

message passing bottleneck and improving per-

formance; and second, we introduce a hierarchi-

cal approach (MultiScale MeshGraphNets) which

passes messages on two different resolutions (ﬁne

and coarse), signiﬁcantly improving the accuracy

of MeshGraphNets while requiring less computa-

tional resources.

1. Introduction

There has been a growing interest in accelerating or replac-

ing costly traditional numerical solvers with learned simula-

tors, which have the potential to be much faster than classi-

cal methods (Thuerey et al.,2020a;Kochkov et al.,2021b;

Pfaff et al.,2021;Keisler,2022). Furthermore, learned sim-

ulators are generally differentiable by construction, which

Equal contribution

DeepMind. Correspondence to: Meire

Fortunato <meirefortunato@google.com>.

2nd AI4Science Workshop at the

39 th

International Conference on

opens up interesting avenues for inverse design (Challapalli

et al.,2021;Goodrich et al.,2021;Allen et al.,2022). A

recent approach to learning simulations discretized on un-

structured meshes is MeshGraphNets (MGN, Pfaff et al.

(2021)), which encodes the simulation mesh at each time

step into a graph, and uses message passing Graph Neu-

ral Networks (GNNs, Gilmer et al. (2017); Scarselli et al.

(2008); Battaglia et al. (2018)) to make predictions on this

graph. MGN demonstrated strong generalization, and accu-

rate predictions on a broad range of physical systems.

The accuracy of traditional solvers is often limited by the

resolution of the simulation mesh. This is particularly true

for chaotic systems like ﬂuid dynamics: processes at very

small length-scales, such as turbulent mixing, affect the

overall ﬂow and need to be resolved on very ﬁne meshes to

accurately solve the underlying partial differential equation.

This leads to the characteristic spatial convergence; where

simulation accuracy increases monotonically with the mesh

resolution. This is an important property for the use of

numerical solvers in practice, as it allows trading in compute

to obtain the desired solution accuracy.

However, it is unclear whether this behavior also applies

to learned simulation approaches, particularly GNN-based

models like MGN. There are reasons to believe it is not

the case: as the mesh becomes ﬁner, message passing

GNNs have to perform more update steps to pass infor-

mation along the same physical distance. This results in

signiﬁcantly higher computational cost, and may also cause

over-smoothing (Li et al.,2018a;Chen et al.,2020).

In this work, we investigate MGN on highly resolved

meshes, and ﬁnd that message propagation speed indeed be-

comes a limiting factor, leading to high computational costs

and reduced accuracy at high resolutions. To overcome this

limitation, we propose two orthogonal approaches:

•

First, we introduce MultiScale MeshGraphNets (MS-

MGN), a hierarchical framework for learning mesh-

based simulations using GNNs, which runs message

passing at two different resolutions. Namely, we have

message passing on input (ﬁne) mesh but also at a

coarser mesh that facilitates the propagation of informa-

tion. We demonstrate that MS-MGN restores spatial

convergence, and is more accurate and computationally

arXiv:2210.00612v1 [cs.LG] 2 Oct 2022

MultiScale MeshGraphNets

efﬁcient than MGN.

•

Second, we modify the training distribution to use high-

accuracy labels that better capture the true dynamics of

the physical system. As opposed to simply replicating

the spatial convergence curve of traditional solvers, this

allows to make better predictions than the reference

simulator at a given resolution.

Together, these approaches are a key step forward for

learned mesh-based simulations, and improve accuracy for

highly resolved simulations at a lower computational cost.

2. MultiScale MeshGraphNets

Here we introduce MultiScale MeshGraphNets (MS-

MGN), a hierarchical version of MeshGraphNets (MGN).

As in MGN, the model uses a message passing GNN to learn

the temporal evolution of physical systems discretized on

meshes. In contrast to MGN, passes are being made on both

the graph deﬁned by the ﬁne input mesh, and in a coarser

mesh. This coarse mesh is introduced only with the aim

of promoting more efﬁcient communication in latent space,

to efﬁciently model fast-acting or non-local dynamics. All

inputs and outputs are deﬁned on the ﬁne input mesh.

This architecture is inspired by both empirical ﬁndings about

message propagation in graphs and by multigrid methods

(Briggs et al.,2000;Bramble,2019). First, message prop-

agation speed in Cartesian coordinates is bounded by the

length of the mesh edges multiplied by the number of mes-

sage passing blocks. Reﬁning the mesh aiming to obtain

greater precision decreases the lengths of the edges, which

implies a lower speed of propagation of information. This

can lead to certain effects not being modeled properly on

high-resolution meshes. Using an auxiliary coarse mesh,

we can retain high message propagation speeds even for

very ﬁne input meshes. And second, GNNs are related to

Gauss-Seidel smoothing iterations as they can only reduce

errors locally. By solving the system at multiple resolutions,

multigrid methods demonstrate an effective way to achieve

global solutions using local updates.

MS-MGN uses the Encode-Process-Decode GNN frame-

work introduced in Sanchez-Gonzalez et al. (2020), and is

trained for next-step predictions and applied iteratively to

unroll trajectories at inference time. For training, encoding

and decoding, we closely follow the MGN architecture. In

this work, we focus on Eulerian dynamics, hence we only

need to consider mesh edges and can omit world edges. The

algorithm is described for 2D triangular meshes but it can

also be applied for e.g. hexahedral or tetrahedral meshes. In

departure from MGN, messages are passed independently

on two graphs, the coarse graph

and ﬁne graph

. Ad-

ditionally, we deﬁne the upsampling and downsampling

Low-resolution

High-resolution

UpsampleDownsample

Figure 1.

The four update operators on MS-MGN:

ownsample

(left), where each node on the low-resolution mesh (orange mesh)

receives information from the high-resolution mesh triangle (blue

mesh) enclosing the node;

igh-resolution (bottom-middle), where

high-resolution nodes are updated by their connected neighbors;

ow-resolution (top-middle), where low-resolution nodes are up-

dated by connected their neighbors;

psample (right), where each

high-resolution node receives information from the corresponding

low-resolution nodes it updates in the Downsample update.

graphs

Gup

Gdown

to propagate information between lev-

els. The training loss is only placed on nodes of the ﬁne

input graph

. Below, we describe graph construction and

message passing operators for these graphs. The four graph

operators are visualized in Figure 1, a more detailed descrip-

tion of encoding and message passing can be found in the

Appendix (A.1).

Encoder

A mesh is an undirected graph

G= (V, E)

speciﬁed by its nodes

and edges

. Let

D ⊂ R2

the physical domain where the problem is deﬁned and let

Gh= (Vh, Eh)

and

Gl= (Vl, El)

denote high-resolution

and low-resolution mesh representations of

, respectively.

We encode the ﬁne input graph

as in Pfaff et al. (2021),

with the same node and edge features, and identical latent

sizes of 128. The coarse graph

is encoded in a similar

fashion. However, we only encode geometric features in the

coarse graph, i.e., relative node coordinates on edges, and

a node type to distinguish between internal and boundary

nodes. The input ﬁeld variables such as velocity are only

encoded into Gh.

We next construct the downsampling graph

Gdown = (Vl∪

Vh, Eh,l)

as follows: For each ﬁne-mesh node

i∈Vh

ﬁnd the triangle on the coarse mesh which contains this node.

Then, we create three edges

kh,l :i→j

which connect the

node

to each corner node

j=j(i)∈Vl

of the triangle.

As the nodes in this graph are already deﬁned above in

Gh,Gl

, we only need to deﬁne the edge feature encoding.

The edge features are the relative node coordinates from

senders and receivers, which are embedded using an MLP

of the same architecture as in the MGN encoders.

For meshes with other element types (e.g. hexahedrons or

tetrahedrons) we can do the same, by ﬁnding the containing ele-

ment and connecting to all corner nodes.

MultiScale MeshGraphNets

The upsampling graph

Gup = (Vl∪Vh, El,h)

has the same

structure. That is, for each

i∈Vl

, we create three edges

kl,h :i→j

connecting

i∈Vl

to the corner nodes of the

triangle in the high-res input mesh that contains the node

Edge features are encoded exactly as in the downsampling

graph. Figure 1shows a representation for both graphs.

Processor

The processor consists of several iterated pro-

cessor blocks, which compute updates on the graphs deﬁned

above by message passing, as in MGN. However, as op-

posed to the single graph in MGN, we now have four graphs

to update. We could construct a processor block which

performs updates on all four graphs simultaneously, and

repeat this block

times. But this would be inefﬁcient: For

example, as inputs and outputs are deﬁned on

only, the

ﬁrst and last updates on the other graphs would be wasted.

We also note that updates on the coarse graph are signif-

icantly cheaper due to the smaller number of nodes and

edges, and propagate information further. Hence, a more

efﬁcient strategy may be to perform a number of updates on

to aggregate local features, downsample to the coarse

graph, perform updates on

, upsample, and perform a few

updates on

to compute small-scale dynamics. This is a

similar strategy as a

-cycle in multigrid methods; and as in

multi-grid methods, we can stack several of these cycles to

reduce errors even further. We will investigate a few choices

for efﬁcient processor architectures in Section 5.

Decoder and state updater.

The next-step state predic-

tions will be produced from the updated node latents on

the ﬁne graph

, and exactly follow the description in

MGN, including the loss function, and hyperparemeters in

the training setup.

3. High-accuracy labels

The prediction quality of a learned model is bounded by the

quality of the data it is trained on.

In Pfaff et al. (2021), the training examples were obtained

by running a traditional reference simulator for a given ini-

tial state and space discretization, and using the simulator’s

predictions as training labels. However, these labels are only

approximations to the actual physical phenomena we are

trying to model, and their accuracy is directly linked to the

spatial resolution of the simulation. To reduce the amount

of computation associated with high-resolution simulations,

traditional solvers often simulate on a coarser grid, and em-

ploy heuristic “closure models” to approximate the effect of

small-scale dynamics below the resolution of the simulation

mesh (Pope,2011).

In a learned model, we have another option: Instead of a

handwritten heuristic, we can modify the training distribu-

tion to use “high-accuracy labels”, e.g. by running a refer-

ence simulation at higher spatial resolution, and bi-linearly

interpolating the solution down to the mesh to be used for

prediction (Figure 7). This way, the model can implicitly

learn the effect of smaller scales without any changes to the

model code, and at inference time potentially achieve solu-

tions which are more accurate than what is possible with a

classical solver on a coarse scale. In Section 5we show that

learning such a subgrid-aware model for ﬂuid dynamics is

indeed tractable over a surprisingly broad resolution scale.

Figure 2.

A reference simulation of the Karman vortex street sim-

ulated with COMSOL. The colormap shows the

-component of

the velocity ﬁeld. Top: The simulation mesh is not ﬁne enough to

resolve all ﬂow features, and the characteristic vortex shedding is

suppressed. Bottom: A more expensive simulation on a ﬁner mesh

correctly resolves the dynamics. Middle: “high-accuracy” labels

from the high-resolution simulation (bottom) interpolated on the

coarse mesh from (top), with vortex-shedding still visible. We use

this to generate a training set.

4. Experimental setup

Training set

We generated a “CylinderFlow” dataset com-

prising

1000

trajectories of incompressible ﬂow past a long

cylinder in a channel, simulated with COMSOL (Comsol,

2020). Each trajectory consists of

T= 200

time steps, and

we vary parameters, such as radius and position of the obsta-

cle, inﬂow initial velocity and mesh resolution. Notably, the

mesh resolution covers a wide range from a hundred to tens

of thousands of mesh nodes. Section A.3 in the Appendix

shows example trajectories and dataset statistics.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MultiScaleMeshGraphNetsMeireFortunato*1TobiasPfaff*1PeterWirnsberger1AlexanderPritzel1PeterBattaglia1AbstractInrecentyears,therehasbeenagrowinginter-estinusingmachinelearningtoovercomethehighcostofnumericalsimulation,withsomelearnedmodelsachievingimpressivespeed-upsoverclassicalsolverswhilstmaintain...

展开>> 收起<<

MultiScale MeshGraphNets Meire Fortunato 1Tobias Pfaff 1Peter Wirnsberger1Alexander Pritzel1Peter Battaglia1 Abstract.pdf

共14页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

MultiScale MeshGraphNets Meire Fortunato 1Tobias Pfaff 1Peter Wirnsberger1Alexander Pritzel1Peter Battaglia1 Abstract

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: