Neuro-Symbolic Partial Differential Equation Solver Pouria A. Mistaniy Nvidia

2025-05-02 0 0 2.47MB 8 页 10玖币

侵权投诉

Neuro-Symbolic Partial Differential Equation Solver

Pouria A. Mistani†∗

Nvidia

pmistani@nvidia.com

Samira Pakravan †

UCSB

spakravan@ucsb.edu

Rajesh Ilango

Nvidia

rilango@nvidia.com

Sanjay Choudhry

Nvidia

schoudhry@nvidia.com

Frederic Gibou

UCSB

fgibou@ucsb.edu

Figure 1: The proposed Neural Bootstrapping Method applied to electrostatics. Left: streamlines.

Right: warped cross-section showing the physically correct jump in the ﬁeld. The neural approach

readily enables a

10243

effective resolution on a single NVIDIA A6000 GPU. Once trained, it takes

sub-milliseconds for the network to evaluate such a simulation [30].

Abstract

We present a highly scalable strategy for developing mesh-free neuro-symbolic

partial differential equation solvers from existing numerical discretizations found in

scientiﬁc computing. This strategy is unique in that it can be used to efﬁciently train

neural network surrogate models for the solution functions and the differential op-

erators, while retaining the accuracy and convergence properties of state-of-the-art

numerical solvers. This neural bootstrapping method is based on minimizing resid-

uals of discretized differential systems on a set of random collocation points with

respect to the trainable parameters of the neural network, achieving unprecedented

resolution and optimal scaling for solving physical and biological systems.

1 Introduction

Most modern physical, biological and engineering systems are described by partial differential

equations on irregular, often moving, boundaries. The difﬁculties in solving those problems stem

from approximating the equations while respecting the physically correct discontinuous nature of the

solution across the boundaries. Smoothing strategies are straightforward to design, but unfortunately

introduce unphysical characteristics in the solution, which lead to systemic errors.

Since the 1990s, artiﬁcial neural networks have been used for solving differential equations using two

general strategies: (i) mapping the algebraic operations of the discretized systems onto specialized

∗corresponding author.

†These authors contributed equally to this work.

36th Conference on Neural Information Processing Systems (NeurIPS 2022).

arXiv:2210.14907v1 [math.NA] 25 Oct 2022

neural network architectures and minimizing the network energy, or (ii) treating the whole neural

network as the basic approximation unit, with parameters trained to minimize a specialized error

function that includes the differential equation itself as well as its boundary and initial conditions.

In the ﬁrst category, neurons output the discretized solution values over a set of grid points, and

minimizing the network energy drives the neuronal values towards the solution at the mesh points.

The neural network energy is the residual of the ﬁnite discretization, summed over all neurons [

]. A

strong feature is the preservation of the ﬁnite discretization convergence; however, the computational

cost grows with increasing resolution and dimensionality. Early examples include [15, 10, 9].

The second strategy, proposed by Lagaris et al. [

], relies on the function approximation capabilities

of the neural networks. Encoding the solution everywhere in the domain within a neural network

offers a mesh-free, compact, and memory efﬁcient surrogate model for the solution function, which

can be used in subsequent inference tasks. This method has recently re-emerged as the physics-

informed neural networks (PINNs) [

] and is widely used. Despite their advantages, these methods

lack the controllable convergence properties of traditional numerical discretizations, and are biased

towards the lower frequency features of the solutions [41, 34, 20].

Hybridization frameworks seek to combine the performance of neural network inference on modern

accelerated hardware with the guaranteed accuracy of traditional discretizations developed by the

scientiﬁc community. The hybridization efforts are algorithmic or architectural.

One important algorithmic method is the deep Galerkin method (DGM) [

], a neural network

extension of the mesh-free Galerkin method where the solution is represented as a deep neural

network rather than a linear combination of basis functions. Being mesh-free, it enables the solution

of high-dimensional problems by training the neural network model to satisfy the differential operator

and its initial and boundary conditions on a randomly sampled set of points, rather than on an

exponentially large grid. Although the number of points can be very large in higher dimensions,

the training is done sequentially on smaller batches of data points and second-order derivatives are

calculated by a scalable Monte Carlo method. Another important algorithmic method is the deep Ritz

[

]. It implements a deep neural network approximation of the trial function that is constrained by

numerical quadrature rules for the variational functional, followed by stochastic gradient descent.

Architectural hybridization is based on differentiable numerical linear algebra. One emerging

class involves implementing differentiable ﬁnite discretization solvers and embedding them in the

neural architectures that enable application of end-to-end differentiable gradient based optimization.

Differentiable solvers have been developed in JAX [

] for ﬂuid dynamic problems, e.g.

Phi-Flow

[

JAX-CFD

[

], and

JAX-FLUIDS

[

]. These methods are suitable for inverse problems where

an unknown ﬁeld is modeled by the neural network, while the model inﬂuence is propagated by the

differentiable solver into a measurable residual [

]. We also note the classic strategy for

solving inverse problems, the adjoint method, to obtain the gradient of the loss without differentiation

across the solver [

]; however, deriving analytic expression for the adjoint equations can be tedious

or impractical. Other notable use of differentiable solvers is to model and correct for the solution

errors of ﬁnite discretizations [40], and learning and controlling differentiable systems [13, 16].

Neural networks are not only universal approximators of continuous functions, but also of nonlin-

ear operators [

]. Although this fact has been leveraged using data-driven strategies for learning

differential operators by many authors [

], researchers have demonstrated the ability

of differentiable solvers to effectively train nonlinear operators without any data in a completely

physics-driven fashion, see section on learning inverse transforms in [33].

We propose a novel framework for solving PDEs based on deep neural networks, that enables lifting

any existing mesh-based ﬁnite discretization method off of its underlying grid and extend it into a

mesh-free and embarrassingly parallel method that can be applied to high dimensional problems on

unstructured random points. In addition, discontinuous solutions can be readily considered.

2 Problem statement

We illustrate our approach by considering a closed irregular interface (

) that partitions an interior

(

Ω−

) and an exterior (

Ω+

) subdomain (see ﬁgure 2). The coupled solution

u±∈Ω±

satisfy the

Helmholtz equation

k±u±− ∇ · (µ±∇u±) = f±

with jump conditions

[u] = α

and

[µ∂nu] = β

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Neuro-SymbolicPartialDifferentialEquationSolverPouriaA.MistaniyNvidiapmistani@nvidia.comSamiraPakravanyUCSBspakravan@ucsb.eduRajeshIlangoNvidiarilango@nvidia.comSanjayChoudhryNvidiaschoudhry@nvidia.comFredericGibouUCSBfgibou@ucsb.eduFigure1:TheproposedNeuralBootstrappingMethodappliedtoelectrostatic...

展开>> 收起<<

Neuro-Symbolic Partial Differential Equation Solver Pouria A. Mistaniy Nvidia.pdf

共8页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Neuro-Symbolic Partial Differential Equation Solver Pouria A. Mistaniy Nvidia

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: