ENFORCING DIRICHLET BOUNDARY CONDITIONS IN PHYSICS -INFORMED NEURAL NETWORKS AND VARIATIONAL PHYSICS -INFORMED NEURAL NETWORKS

2025-05-06 0 0 5.77MB 22 页 10玖币

侵权投诉

ENFORCING DIRICHLET BOUNDARY CONDITIONS IN

PHYSICS-INFORMED NEURAL NETWORKS AND VARIATIONAL

PHYSICS-INFORMED NEURAL NETWORKS

A PREPRINT

S. Berrone∗C. Canuto∗M. Pintore∗N. Sukumar†

August 1, 2023

ABSTRACT

In this paper, we present and compare four methods to enforce Dirichlet boundary conditions in

Physics-Informed Neural Networks (PINNs) and Variational Physics-Informed Neural Networks

(VPINNs). Such conditions are usually imposed by adding penalization terms in the loss function

and properly choosing the corresponding scaling coefﬁcients; however, in practice, this requires an

expensive tuning phase. We show through several numerical tests that modifying the output of the

neural network to exactly match the prescribed values leads to more efﬁcient and accurate solvers.

The best results are achieved by exactly enforcing the Dirichlet boundary conditions by means of

an approximate distance function. We also show that variationally imposing the Dirichlet boundary

conditions via Nitsche’s method leads to suboptimal solvers.

Keywords Dirichlet boundary conditions, PINN, VPINN, deep neural networks, approximate distance function

2020 MSC 35A15, 65L10, 65L20, 65K10, 68T05

1 Introduction

Physics-Informed Neural Networks (PINNs), proposed in [

] after the initial pioneering contributions of Lagaris et

al. [

], are rapidly emerging computational methods to solve partial differential equations (PDEs). In its basic

formulation, a PINN is a neural network that is trained to minimize the PDE residual on a given set of collocation points

in order to compute a corresponding approximate solution. In particular, the fact that the PDE solution is sought in a

nonlinear space via a nonlinear optimizer distinguishes PINNs from classical computational methods. This provides

PINNs ﬂexibility, since the same code can be used to solve completely different problems by adapting the neural network

loss function that is used in the training phase. Moreover, due to the intrinsic nonlinearity and the adaptive architecture

of the neural network, PINNs can efﬁciently solve inverse [

], parametric [

], high-dimensional [

] as

well as nonlinear [

] problems. Another important feature characterizing PINNs is that it is possible to combine

distinct types of information within the same loss function to readily modify the optimization process. This is useful,

for instance, to effortlessly integrate (synthetic or experimental) external data into the training phase to obtain an

approximate solution that is computed using both data and physics [9].

In order to improve the original PINN idea, several extensions have been developed. Some of these developments

include the Deep Ritz method (DRM) [

], in which the energy functional of a variational problem is minimized; the

conservative PINN (cPINN) [

], where the approximate solution is computed by a domain-decomposition approach

enforcing ﬂux conservation at the interfaces, as well as its improvement in the extended PINN (XPINN) [

]; and the

variational PINN (VPINN) [22, 23], in which the loss function is deﬁned by exploiting the variational structure of the

underlying PDE.

∗

Dipartimento di Scienze Matematiche, Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129 Torino, Italy. ste-

fano.berrone@polito.it (S. Berrone), claudio.canuto@polito.it (C. Canuto), moreno.pintore@polito.it (M. Pintore).

†

Department of Civil and Environmental Engineering, University of California, Davis, CA 95616, USA. nsukumar@ucdavis.edu

arXiv:2210.14795v2 [math.NA] 31 Jul 2023

Enforcing Dirichlet boundary conditions in PINNs and VPINNs A PREPRINT

Most of the existing PINN approaches enforce the essential (Dirichlet) boundary conditions by means of additional

penalization terms that contribute to the loss function, these are each multiplied by constant weighting factors. See

for instance [

]; note that this list is by no means exhaustive, therefore we also refer

to [

] for more detailed overviews of the PINN literature. However, such an approach may lead to poor

approximation, and therefore several techniques to improve it have been proposed. In [

] and [

], adaptive scaling

parameters are proposed to balance the different terms in the loss functions. In particular, in [

] the parameters are

updated during the minimization to maximize the loss function via backpropagation, whereas in [

] a ﬁxed learning

rate annealing procedure is adopted. Other alternatives are related to adaptive sampling strategies (e.g., [

]) or

to speciﬁc techniques such as the Neural Tangent Kernel [44].

Note that although it is possible to automatically tune these scaling parameters during the training, such techniques

require more involved implementations and in most cases lead to intrusive methods since the optimizer has to be

modiﬁed. Instead, in this paper, we focus on three simple and non-intrusive approaches to impose Dirichlet boundary

conditions and we compare their accuracy and efﬁciency. The proposed approaches are tested using standard PINN and

interpolated VPINN which have been proven to be more stable than standard VPINNs [6].

The main contributions of this paper are as follows:

We present three non-standard approaches to enforce Dirichlet boundary conditions on PINNs and VPINNs,

and discuss their mathematical formulation and their pros and cons. Two of them, based on the use of an

approximate distance function, modify the output of the neural network to exactly impose such conditions,

whereas the last one enforces them approximately by a weak formulation of the equation.

The performance of the distinct approaches to impose Dirichlet boundary conditions is assessed on various

test cases. On average, we ﬁnd that exactly imposing the boundary conditions leads to more efﬁcient and

accurate solvers. We also compare the interpolated VPINN to the standard PINN, and observe that the different

approaches used to enforce the boundary conditions affect these two models in similar ways.

The structure of the remainder of this paper is as follows. In Section 2, the PINN and VPINN formulations are described:

ﬁrst, we describe the neural network architecture in Section 2.1 and then focus on the loss functions that characterize

the two models in Section 2.2. Subsequently, in Section 3, we present the four approaches to enforce the imposition

of Dirichlet boundary conditions; three of them can be used with both PINNs and VPINNs, whereas the last one is

used to enforce the required boundary conditions only on VPINNs because it relies on the variational formulation.

Numerical results are presented in Section 4. In Section 4.1, we ﬁrst analyze for a second-order elliptic problem the

convergence rate of the VPINN with respect to mesh reﬁnement. In doing so, we demonstrate that when the neural

network is properly trained, identical optimal convergence rates are realized by all approaches only if the PDE solution

is simple enough. Otherwise, only enforcing the Dirichlet boundary conditions with Nitsche’s method or by exactly

imposing them via approximate distance functions ensure the theoretical convergence rate. In addition, we compare

the behavior of the loss function and the

error while increasing the number of epochs, as well as the behavior of

the error when the network architecture is varied. In Section 4.2, we show that it is also possible to efﬁciently solve

second-order parametric nonlinear elliptic problems. Furthermore, in Sections 4.3– 4.5, we compare the performance of

all approaches on PINNs and VPINNs by solving a linear elasticity problem and a stabilized Eikonal equation over an

L-shaped domain, and a convection problem. Finally, in Section 5, we close with our main ﬁndings and present a few

perspectives for future work.

2 PINNs and interpolated variational PINNs

In this section, we describe the PINN and VPINN that are used in Section 4. In particular, in Section 2.1 the neural

network architecture is presented, and the construction of the loss functions is discussed in Section 2.2.

2.1 Neural network description

In this work we compare the efﬁciency of four approaches to enforce Dirichlet boundary conditions in PINN and

VPINN. The main difference between these two numerical models is the training loss function; the architecture of the

neural network is the same and is independent of the way the boundary conditions are imposed.

In our numerical experiments we only consider fully-connected feed forward neural networks with a ﬁxed architecture.

Such neural networks can be represented as nonlinear parametric functions

uNN :RNin →RNout

that can be evaluated

via the following recursive formula:

x∗

i=σiAix∗

i−1+bi, i = 1,2, . . . , L. (2.1)

Enforcing Dirichlet boundary conditions in PINNs and VPINNs A PREPRINT

In particular, with the notation of

(2.1)

x∗

0∈RNin

is the neural network input vector,

x∗

L∈RNout

is the neural network

output vector, the neural network architecture consists of an input layer,

L−1

hidden layers and one output layer,

and

are matrices and vectors containing the neural network weights, and

σi:R→R

is the activation function of the

-th layer and is element-wise applied to its input vector. We also remark that the

-th layer is said to contain

dim(x∗

neurons and that

σi

has to be nonlinear for any

i= 1,2, . . . , L −1

. Common nonlinear activation functions are the

rectiﬁed linear unit (

ReLU(x) := max(0, x)

), the hyperbolic tangent and the sigmoid function. In this work, we take

σLto be the identity function in order to avoid imposing any constraint on the neural network output.

The weights contained in

and

can be logically reorganized in a single vector

wNN

. The goal of the training phase

is to ﬁnd a vector

wNN

that minimizes the loss function; however, since such a loss function is nonlinear with respect

to wNN and the corresponding manifold is extremely complicated, we can at best ﬁnd good local minima.

2.2 PINN and interpolated VPINN loss functions

For the sake of simplicity, the loss function for PINN and interpolated VPINN is stated for second-order elliptic

boundary-value problems. However, the discussion can be directly generalized to different PDEs, and in Section 4,

numerical results associated with other problems are also presented.

Let us consider the model problem:







Lu := −∇ · (µ∇u) + β· ∇u+σu =fin Ω,

u=gon ΓD,

µ∂u

∂n =ψon ΓN,

(2.2)

where

Ω⊂Rn

is a bounded domain whose Lipschitz boundary

∂Ω

is partitioned as

∂Ω = ΓD∪ΓN

, with

measn−1(ΓD)>0

. For the well-posedness of the boundary-value problem we require

σ∈L∞(Ω)

and

β∈(W1,∞(Ω))n

satisfying, in the entire domain

Ω

µ≥µ0

for some strictly positive constant

µ0

and

σ−1

2∇·β≥0

Moreover,

f∈L2(Ω)

ψ∈L2(ΓN)

and

g=u|ΓD

for some

u∈H1(Ω)

. We point out that even if these assumptions

ensure the well-posedness of the problem, PINNs and VPINNs often struggle to compute low regularity solutions. We

refer to [42] for a recent example of a neural network based model that overcomes this issue.

In order to train a PINN, one introduces a set of collocation points

{x1, . . . , xNI}

and evaluates the corresponding

equation residuals {rPINN

1, . . . , rPINN

NI}. Such residuals, for problem (2.2), are deﬁned as:

rPINN

i(u) = −∇ · (µ∇u)(xi) + β· ∇u(xi) + σu(xi)−f(xi)∀i= 1,2, . . . , NI.(2.3)

Since we are interested in a neural network that satisﬁes the PDE in a discrete sense, the loss function minimized during

the PINN training is:

PINN(w) =

i=1 rPINN

i(w)

2.(2.4)

(2.4)

, when

is sufﬁciently large and

PINN(uNN)

is close to zero, the function

uNN

represented by the neural

network output approximately satisﬁes the PDE and can thus be considered a good approximation of the exact solution.

Other terms are often added to impose the boundary conditions or improve the training, which are discussed in Section 3.

Let us now focus on the interpolated VPINN proposed in [

]. We introduce the function spaces

U:= H1(Ω)

and

V:= {v∈H1(Ω) : v|ΓD= 0}, the bilinear form a:U×V→Rand the linear form F:V→R,

a(w, v) = ZΩ

µ∇w· ∇v+β∇wv +σwv, F (v) = ZΩ

fv +ZΓN

ψv.

The variational counterpart of problem (2.2) thus reads: Find u∈Usuch that:

a(u, v) = F(v)∀v∈V,

u=gon ΓD.(2.5)

In order to discretize problem

(2.5)

, we use two discrete function spaces. Inspired by the Petrov-Galerkin framework,

we denote the discrete trial space by

Uh⊂U

and the discrete test space by

Vh⊂V

. The functions comprising such

spaces are generated on two conforming, shape-regular and nested partitions

and

with compatible meshsizes

and

, respectively. Assuming that

is the ﬁner mesh, one can claim that

H≲h < H

and that every element of

strictly contained in an element of TH.

Enforcing Dirichlet boundary conditions in PINNs and VPINNs A PREPRINT

0 0.2 0.4 0.6 0.8 1

0.5

(a)

0 0.2 0.4 0.6 0.8 1

-0.5

0.5

(b) (c)

Fig. 1 Pair of meshes and corresponding basis functions of a one-dimensional discretization (left) and nested meshes

and

in a two-dimensional domain (right). (a) Basis functions of

. The ﬁlled circles (red) are the nodes of

the corresponding mesh

; (b) Basis functions of

. The ﬁlled circles (blue) are the vertex nodes that deﬁne the

elements of the corresponding mesh

; and (c) Meshes used in the numerical experiments of Sections 4.3 and 4.4.

The blue mesh is TH, the red one is Th. All the ﬁgures are obtained with q= 3,ktest = 1,kint = 4.

Denoting by

UH:= span{φu

i:i∈IH} ⊂ U

the space of piecewise polynomial functions of order

kint

over

and

Vh:= span{φv

i:i∈Ih} ⊂ V

the space of piecewise polynomial functions of order

ktest

over

that vanish on

ΓD

we deﬁne the discrete variational problem as: Find u∈UHsuch that:

a(u, v) = F(v)∀v∈Vh,

u=gHon ΓD,(2.6)

where

is a suitable piecewise polynomial approximation of

. A representation of the spaces

and

in a

one-dimensional domain is provided in Figures 1a and 1b. Examples of pair of meshes

and

are shown in Fig. 1c.

In order to obtain computable forms

and

, we introduce elemental quadrature rules of order

and deﬁne

ah(·,·)

and

Fh(·)

as the approximations of

a(·,·)

and

F(·)

computed with such quadrature rules. In [

], under suitable

assumptions, an a priori error estimate with respect to mesh reﬁnement has been proved when

q=kint +ktest −2

. It is

then possible to deﬁne the computable variational residuals associated with the basis functions of Vhas:

rh,i(w) = Fh(φv

i)−ah(w, φv

i), i ∈Ih.(2.7)

Consequently, in order to compute an approximate solution of problem

(2.6)

, one seeks a function

w∈UH

that

minimizes the quantity:

h(w) = X

i∈Ih

h,i(w),(2.8)

and satisﬁes the imposed boundary conditions. We refer to Section 3 for a detailed description of different approaches

used to impose Dirichlet boundary conditions. It should be noted that, since in Sections 4.2–4.5 we consider problems

other than

(2.2)

, the residuals in

(2.7)

have to be suitably modiﬁed, while the loss function structure deﬁned in

(2.8)

maintained.

We are interested in using a neural network to ﬁnd the minimizer of

. We thus denote by

IH:C0(Ω) →UH

interpolation operator used to map the function

uNN

associated with the neural network to its interpolating element in

, and train the neural network to minimize the quantity

h(IHuNN )

. We highlight that in order to construct the

function

IHuNN

, the neural network has to be evaluated only on

dim(UH)

interpolation points

{xI

1, . . . , xI

dim(UH)} ⊂ Ω

Then, assuming that {φu

i:i∈IH}is a Lagrange basis such that φu

i(xI

j) = δij for every i, j ∈IH, it holds:

IHuNN =X

i∈IH

uNN (xI

i)φu

i.(2.9)

Enforcing Dirichlet boundary conditions in PINNs and VPINNs A PREPRINT

We remark that the approaches proposed in Section 3 can also be used on non-interpolated VPINNs. However, we

restrict our analysis to interpolated VPINNs because of their better stability properties (see Fig. 11 and the corresponding

discussion).

3 Mathematical formulation

We compare four methods to impose Dirichlet boundary conditions on PINNs and VPINNs. We do not consider

Neumann or Robin boundary conditions since they can be weakly enforced by the trained VPINN due to the chosen

variational formulation (computations using PINNs is discussed in [

]). We also highlight that method

below

can be used only with VPINNs because it relies on the variational formulation of the PDE. We analyze the following

methods:

MA:

Incorporation of an additional cost in the loss function that penalizes unsatisﬁed boundary conditions; this is

the standard approach in PINNs and VPINNs because of its simplicity and effectiveness. In fact, it is possible

to choose

control points

{xg

1, . . . , xg

NB} ⊂ ΓD

and modify the loss functions deﬁned in

(2.4)

(2.8)

follows:

PINN(w) =

i=1 rPINN

i(w)

2+λ

i=1

(w(xg

i)−g(xg

i))2,(3.1)

h(w) = X

i∈Ih

h,i(w) + λ

i=1

(w(xg

i)−g(xg

i))2,(3.2)

where

λ > 0

is a model hyperparameter. Note that on considering the interpolated VPINN and exploiting the

solution structure in

(2.9)

, it is possible to ensure the uniqueness of the numerical solution by choosing the

control points {xg

1, . . . , xg

NB}as the NBinterpolation points belonging to ΓD.

We also highlight that such a method can be easily adapted to impose other types of boundary conditions just

by adding suitable terms to

(3.1)

and

(3.2)

. On the other hand, despite its simplicity, the main drawback of this

approach is that it leads to a more complex multi-objective optimization problem.

MB:

Exactly imposing the Dirichlet boundary conditions as described in [

] and [

]. In this method we add a

non-trainable layer Bat the end of the neural network to modify its output waccording to the rule:

Bw =g+ϕw, (3.3)

where

g∈C0(Ω)

is an extension of the function

inside the domain

Ω

(i.e.,

g|ΓD=g

) and

ϕ∈C0(Ω)

an approximate distance function (ADF) to the boundary

ΓD

, i.e.,

ϕ(x)=0

if and only if

x∈ΓD

, and it is

positive elsewhere. During the training phase one minimizes the quantity R2

PINN(Bw)or R2

h(Bw).

For the sake of simplicity, we only consider ADFs for two-dimensional unions of segments, even though the

approach generalizes to more complex geometries. Following the derivation of

and

in [

], we start by

deﬁning

as the signed distance function from

x:= (x, y)

to the line deﬁned by the segment

of length

with vertices A= (xA, yA)and B= (xB, yB):

d(x) = (x−xA)(yB−yA)−(y−yA)(xB−xA)

Then, we denote

(xc, yc) := (xA+xB)/2,(yA+yB)/2

to be the center of

and deﬁne

as the following

trimming function:

t(x) = 1

L"L

22

− ∥(x, y)−(xc, yc)∥2#.

Note that t≥0deﬁnes a circle of center (xc, yc). Finally, the ADF to AB is deﬁned as

ϕ(x) = v

td2+ √t2+d4−t

2!2

A graphical representation of

d(x)

t(x)

and

ϕ(x)

for an inclined line segment is shown in Figures 2a, 2b

and2c, respectively.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ENFORCINGDIRICHLETBOUNDARYCONDITIONSINPHYSICS-INFORMEDNEURALNETWORKSANDVARIATIONALPHYSICS-INFORMEDNEURALNETWORKSAPREPRINTS.Berrone∗C.Canuto∗M.Pintore∗N.Sukumar†August1,2023ABSTRACTInthispaper,wepresentandcomparefourmethodstoenforceDirichletboundaryconditionsinPhysics-InformedNeuralNetworks(PINNs)and...

展开>> 收起<<

ENFORCING DIRICHLET BOUNDARY CONDITIONS IN PHYSICS -INFORMED NEURAL NETWORKS AND VARIATIONAL PHYSICS -INFORMED NEURAL NETWORKS.pdf

共22页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

ENFORCING DIRICHLET BOUNDARY CONDITIONS IN PHYSICS -INFORMED NEURAL NETWORKS AND VARIATIONAL PHYSICS -INFORMED NEURAL NETWORKS

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: