Towards a machine learning pipeline in reduced order modelling for inverse problems neural networks for boundary parametrization dimensionality reduction and solution

2025-05-06 2 0 3.85MB 21 页 10玖币

侵权投诉

Towards a machine learning pipeline in reduced order

modelling for inverse problems: neural networks for boundary

parametrization, dimensionality reduction and solution

manifold approximation

Anna Ivagnes∗

, Nicola Demo†

, and Gianluigi Rozza‡

Mathematics Area, mathLab, SISSA, via Bonomea 265, I-34136 Trieste, Italy

Abstract

In this work, we propose a model order reduction framework to deal with inverse prob-

lems in a non-intrusive setting. Inverse problems, especially in a partial diﬀerential equation

context, require a huge computational load due to the iterative optimization process. To accel-

erate such a procedure, we apply a numerical pipeline that involves artiﬁcial neural networks

to parametrize the boundary conditions of the problem in hand, compress the dimensionality

of the (full-order) snapshots, and approximate the parametric solution manifold. It derives a

general framework capable to provide an ad-hoc parametrization of the inlet boundary and

quickly converges to the optimal solution thanks to model order reduction. We present in this

contribution the results obtained by applying such methods to two diﬀerent CFD test cases.

1 Introduction

Inverse problems is a wide family of problems that crosses many diﬀerent sciences and engineering

ﬁelds. Inverse problems aim to compute from the given observations the cause that has produced

them, as also explained in [3, 32]. Formally, we can consider an input Iand an output O, and

suppose that there exists a map

M:i→o

that models a mathematical or physical law. The computation of the output as o=M(i) is called

direct problem, whereas the problem of ﬁnding the input given the output is called inverse problem.

Given a certain output ot, the inverse problem consists of inverting the map Mand ﬁnding the

input itwhich produces the output ot, i.e., which satisﬁes M(it) = ot. Inverse problems may be of

interest for a lot of mathematical ﬁelds, from heat transfer problems to ﬂuid dynamics. The case

study which is here analysed is a Navier-Stokes ﬂow in a circular cylinder, and the aim is to ﬁnd the

proper boundary condition in order to obtain the expected distribution within the domain. Such

an application tries to solve a typical naval problem, numerically looking for the inlet setting which

provides the right working condition during the optimization of the propulsion system. Propeller

optimization is indeed very sensitive to modiﬁcations in the velocity distribution at the propeller

plane: to obtain an optimized artifact it becomes very important to set up the numeric simulation

such that the velocity distribution is as close as possible to the target distribution, usually collected

by experimental tests. The problem is the identiﬁcation of the inlet boundary condition, given the

velocity distribution at the so-called wake, which is the plane (or a portion of it) orthogonal to the

cylinder axis where the propeller operates. To achieve that, the inlet distribution is searched by

parametrizing the target wake — by exploiting a neural network, as we will describe in the next

paragraphs — and optimizing these parameters such that in the simulation the velocity is close to

the target wake. It must be said that to produce meaningful results, we assume here the ﬂow has a

main direction that is perpendicular to the inlet and wake planes: in such a way, the distributions

∗anna.ivagnes@sissa.it

†nicola.demo@sissa.it

‡gianluigi.rozza@sissa.it

arXiv:2210.14764v1 [math.NA] 26 Oct 2022

at these planes are similar to each other, allowing us to search for the optimal inlet among the

parametrized wake distributions. Even if in this case the numerical experiments are conducted in a

Computational Fluid Dynamics (CFD) setting, the methodology is in principle easily transferable

to diﬀerent contexts.

Typically, such an operation is performed within an optimization procedure, in which the di-

rect problem is iteratively solved by varying the input until the desired output is reached. This,

of course, implies the necessity to numerically parametrize the input in a proper way, possibly

allowing a large variety of admissible inputs and at the same time a limited number of parameters.

Moreover, the necessity to solve the direct problem for many diﬀerent instances makes the entire

process computationally expensive, especially dealing with the numerical solution of Partial Dif-

ferential Equations (PDEs). A possible solution to overcome such computational burden is oﬀered

by the Reduced Order Modelling (ROM) techniques.

ROM constitutes a constantly growing approach for model simpliﬁcation, allowing for a real-

time approximation of the numerical solution of the problem at hand. Among the methods al-

ready introduced in the literature, the Proper Orthogonal Decomposition (POD) has become

in recent developments an important tool for dealing with PDEs, especially in parametric set-

tings [37, 29, 13, 35]. Such a framework aims to eﬃciently combine the numerical solutions for

diﬀerent conﬁgurations of the problem, typically pre-computed using consolidated methods — e.g.

ﬁnite volume, ﬁnite element — such that at any model inference all this information is combined

for providing a fast approximation. Within iterative and many-query processes, like inverse prob-

lems, this methodology allows a huge computational gain. The many repetitions of the direct

problem, needed to ﬁnd the target input, can be performed at the reduced level, requiring then a

ﬁnite and ﬁxed set of numerical solutions only for building the ROM. The coupling between ROM

and the inverse problem has been already investigated in literature for heat ﬂux estimation in a

data assimilation context [25], in aerodynamic application [2], in haemodynamic problems [19].

An alternative way to eﬃciently deal with this kind of problem has been explored in a Bayesian

framework [22]. Moreover, among all the contributions in literature we cite [40, 16] as an example

of inverse problem with pointwise observations and inverse problem in a boundary element method

context, respectively.

This contribution introduces an entire and novel machine learning pipeline to deal with the

inverse problems in a ROM setting. In speciﬁc, we combine three diﬀerent uses of Artiﬁcial Neural

Network (ANN), that are: i) parametrization of the boundary condition given a certain target

distribution or pointwise observations, ii) dimensionality compression of the discrete space of the

original — the so-called full-order — model and iii) approximation of the parametric solution

manifold. It derives a data-driven pipeline (graphically represented in Figure 1) able to provide a

parametrization of the original problem, which is successively exploited for the optimization in the

reduced space. Finally, the optimization is carried out by involving a Genetic Algorithm (GA),

but in principle can be substituted by any other optimization algorithm.

The contribution presents in Section 2 an algorithmic overview of the employed methods,

whereas Section 3 illustrates the results of the numerical investigation pursued to the above-

mentioned test case. In particular, we present details for all the intermediate outcomes, comparing

them to the results obtained by employing state-of-the-art techniques. Finally, Section 4 is ded-

icated to summarizing the entire content of the contribution, drawing future perspectives and

highlighting the criticisms highlighted during the development of this contribution.

2 Methodology

We dedicate this section to providing an algorithmic overview of the numerical tools composing

the computational pipeline.

2.1 Boundary parametrization using ANN

Neural networks are a class of regression techniques and the general category of Feed-forward neu-

ral networks has been the subject of considerable research in several ﬁelds in recent years. The

capability of ANN to approximate any function [15] and the even greater computational availability

allowed indeed the massive employment of such an approach to overcome many limitations. In the

Figure 1: Flow diagram for the data-driven pipeline followed in the paper.

ﬁeld of PDEs, we cite [30, 4, 12, 20, 28, 23, 41, 27] as some of main impacting frameworks recently

explored. A not yet investigated usage of ANN, to the best of authors’ knowledge, is instead the

parametrization of a given (scattered) function. We suppose that we have a target distribution

vtarget = (vtarget

i)P

i=1, corresponding to the wake velocity distribution in our case, which has Pde-

grees of freedom. We want to reproduce this distribution by exploiting a neural network technique.

A neural network is deﬁned as a concatenation of an input layer, multiple hidden layers, and

a ﬁnal layer of output neurons. An output of the i-th neuron in the l-th layer of the network is

generally deﬁned as:

i=σ



Nl−1

j=1

ij hl−1

j+bl

i

, i = 1, . . . , Nl, l = 1, . . . , H . (1)

where σis the activation function (which provides non-linearity), bl

ithe bias and Wrefers to the

so-called weights of the synapses of the network, Nlis the number of neurons of the l-th hidden

layer, His the number of layers of the network.

The bias and the weights are the hyperparameters of the network, that are tuned during

the training procedure to converge to the optimal values for the approximation in hand. We

can think of these hyperparameters as the degree of freedom of our approximation, allowing us

to manipulate such distribution by perturbating them. We deﬁne the optimal hyperparameters

(computed with a generic optimization procedure [36, 33]) as b∗and W∗; the network exploiting

these hyperparameters reproduces as output an approximation of our target wake distribution:

N(x)'vtarget(x).

The input xto the whole neural network in this paper corresponds to the polar coordinates of the

points of the wake, so we have a two-dimensional input. We can then rearrange Eq. 1 to express

the parametrized output of a single hidden layer as follows:

i(µ) = σ



Nl−1

j=1

W∗l

ij hl−1

j+ (b∗l

i+µl

i)

, i = 1, . . . , Nl, l = 1, . . . , H. (2)

where µlis the vector of parameters in layer l, which applies only to the bias of the layers. We

ﬁnally obtain the parametrized output N(x,µ).

The main advantage of this approach is the capability to parametrize any initial distribution,

since the weights are initially learnt during the training and then manipulated to obtain its para-

metric version. On the other hand, the dimension of the weights is typically very large, especially

in networks with more than one layer. In networks composed of a large number of layers, a high

number of hyperparameters should be tuned. A possible solution to overcome such an issue could

be to manipulate just a subset of all the hyperparameters. Indeed, in this paper, only the bias

parameters of two hidden layers are perturbed.

Such a posteriori parametrization of any generic distribution is employed in this work to deal

with the inverse problem. The main idea is to compute diﬀerent inlet velocity distributions cor-

responding to diﬀerent weights of the ANN. The weights perturbations are used as parameters

to build the reduced order model, providing an ad-hoc parametrization of the boundary condition

based on the target output. It must be said that such parametrization may lead to good results

only if some correlation between the boundaries and the target output exists.

2.2 Model Order Reduction

ROMs are a class of techniques aimed to reduce the computational complexity of mathematical

models.

The technique used in this paper is data-driven or non-intrusive ROM, which allows us to

build a reduced model without requiring the knowledge of the governing equations of the observed

phenomenon. For this reason, this technique is suitable to deal with experimental data and it

has been widely used in numerical simulations for industrial, naval, biomedical, and environmental

applications [38, 5, 39, 10, 8, 42, 11].

Non-intrusive ROMs are based on an oﬄine-online procedure. Each stage of the procedure can

be approached in diﬀerent ways, which are analyzed in the following Sections. All the techniques

presented in the next lines have been implemented in the Python package called EZyRB [7].

2.2.1 Dimensionality reduction

In the dimensionality reduction step, a given set of high-dimensional snapshots is represented by a

few reduced coordinates, in order to ﬁght the curse of dimensionality and make the pipeline more

eﬃcient.

We consider the following matrix of Msnapshots, collecting our data:

Y=



| | | |

vwake

1vwake

2vwake

3... vwake

| | | | 

=





| | | |

vwake(µ1)vwake(µ2)vwake(µ3). . . vwake(µM)

| | | | 

,

where vwake

i∈RP,i= 1, . . . , M are the velocity distributions in our case, each one corresponding

to a diﬀerent set of parameters µi∈Rpfor i= 1, . . . , M. The goal is to calculate the reduced

snapshots {ˆ

vwake

i∈RL}M

i=1 such that

vwake

i'Ψ−1(Ψ(vwake

i)),ˆ

vwake

i= Ψ(vwake

i),(3)

where Ψ : RP→RL. We specify that the latent dimension LPhas to be selected a priori.

Such phase can be approached by making use of linear or non-linear techniques, such as the

POD and the usage of an Autoencoder (AE), respectively.

Proper Orthogonal Decomposition In the ﬁrst case, the oﬄine part consists of the com-

putation of a reduced basis, composed of a reduced number of vectors named modes. The main

assumption on which it is based is that each snapshot can be approximated by a linear combination

of modes:

vwake

k=1

iϕk, L P , i = 1, . . . , M,

with ϕi∈RPare the modes and ak

iare the related modal coeﬃcients.

The computation of modes in the POD procedure can be done in diﬀerent ways, such as via

Singular Value Decomposition (SVD) or the velocity correlation matrix. In the ﬁrst case, the

matrix Ycan be decomposed via singular value decomposition in the following way:

Y=UΣVT,

where the matrix U∈RP×Lstores the POD modes in its columns and matrix Σ∈RL×Lis a

diagonal matrix including the singular values in descending order. The matrix of modal coeﬃcients

— so, the reduced coordinates — in this case can be computed by:

Y=UTY,

where the ˆ

Y∈RL×Mcolumns are the reduced snapshots. In the second case, the POD space

VP OD = span{[φi]L

i=1}is found solving the following minimization problem:

VP OD = arg minn=1,...,M

n=1

kYn−

i=1

nϕik2∀n= 1, . . . , M,

where Ynis the n-th column of the matrix Y. This problem is equivalent to computing the

eigenvectors and the eigenvalues of the velocity correlation matrix:

(C)ij = (Yi,Yu)L2(Ω),

where Ω is the domain on which the velocity is deﬁned (the propeller wake plane in this case). The

POD modes are expressed as:

ϕi=1

Mλu

j=1

YjVy

ij ,

where Vystores the eigenvectors of the correlation matrix in its columns and λy

iare its eigenvalues.

Autoencoder AE refers to a family of neural networks that, for its architectural features, has

become a mathematical tool for dimensionality reduction [20]. In general, an AE is a neural

network that is composed of two main parts:

•the encoder: a set of layers that takes as input the high-dimensionality vector(s) and returns

the reduced vector(s).

•the decoder, on the other hand, computes the opposite operation, returning a high-dimensional

vector by passing as input the low-dimensional one.

The layers composing the AE could be in principle of any type — e.g. convolutional [34], dense

— but in this work both the encoder and the decoder are feed-forward neural networks. For sake

of simplicity, we assume here that both the encoder and the decoder are built with only one hidden

layer and we denote by Dthe decoder and with Ethe encoder. If vwake is the input of the AE, we

denote by ˜

vwake = (D ◦ E)(vwake) = AE(vwake) the output of the encoder, where formally:

E(vwake) = σ(Wvwake +b) = ˆ

vwake,

D(ˆ

vwake) = σ(W0ˆ

vwake +b0) = ˜

vwake.

A generic structure of an autoencoder is schematized in Figure 2. Weights and activation

functions can be diﬀerent for encoder and decoder, and of course the case with more than one

hidden layer for the encoder and the decoder can be easily derived from this formulation. The AE

is trained by minimizing the following loss function:

kvwake −˜

vwakek2=kvwake −σ0(W0(σ(Wvwake +b)) + b0)k2,

where vwake is the original (high-dimensional) vector to reduce, ˆ

vwake represents the reduced co-

ordinates and ˜

vwake is the predicted vector. In this way, the network weights are optimized such

that the entire AE produces the approximation of the original vector, but compressing it onto

an a priori reduced dimension. The learning step aims so to discover the latent dimension of the

problem at hand.

For what concerns the test cases here considered, two diﬀerent types of autoencoders are taken

into account:

(i) a linear autoencoder, i.e. without an activation function between the hidden layers, with

a single hidden layer composed of a number of neurons equal to the reduced dimension: it

should exactly reproduce the behavior of the POD;

(ii) a non-linear autoencoder, i.e. with an activation function, and with multiple hidden layers,

whose performance is compared to that of the POD.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Towardsamachinelearningpipelineinreducedordermodellingforinverseproblems:neuralnetworksforboundaryparametrization,dimensionalityreductionandsolutionmanifoldapproximationAnnaIvagnes*,NicolaDemo,andGianluigiRozzaMathematicsArea,mathLab,SISSA,viaBonomea265,I-34136Trieste,ItalyAbstractInthiswork,wepro...

展开>> 收起<<

Towards a machine learning pipeline in reduced order modelling for inverse problems neural networks for boundary parametrization dimensionality reduction and solution.pdf

共21页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Towards a machine learning pipeline in reduced order modelling for inverse problems neural networks for boundary parametrization dimensionality reduction and solution

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: