Digital twins of nonlinear dynamical systems Ling-Wei Kong1Yang Weng1Bryan Glaz2Mulugeta Haile2and Ying-Cheng Lai1 3 1School of Electrical Computer and Energy Engineering

2025-04-26 0 0 3.92MB 21 页 10玖币

侵权投诉

Digital twins of nonlinear dynamical systems

Ling-Wei Kong,1Yang Weng,1Bryan Glaz,2Mulugeta Haile,2and Ying-Cheng Lai1, 3, ∗

1School of Electrical, Computer and Energy Engineering,

Arizona State University, Tempe, Arizona 85287, USA

2Vehicle Technology Directorate, CCDC Army Research Laboratory,

2800 Powder Mill Road, Adelphi, MD 20783-1138, USA

3Department of Physics, Arizona State University, Tempe, Arizona 85287, USA

(Dated: October 13, 2022)

We articulate the design imperatives for machine-learning based digital twins for nonlinear dy-

namical systems subject to external driving, which can be used to monitor the “health” of the target

system and anticipate its future collapse. We demonstrate that, with single or parallel reservoir com-

puting conﬁgurations, the digital twins are capable of challenging forecasting and monitoring tasks.

Employing prototypical systems from climate, optics and ecology, we show that the digital twins

can extrapolate the dynamics of the target system to certain parameter regimes never experienced

before, make continual forecasting/monitoring with sparse real-time updates under non-stationary

external driving, infer hidden variables and accurately predict their dynamical evolution, adapt to

diﬀerent forms of external driving, and extrapolate the global bifurcation behaviors to systems of

some diﬀerent sizes. These features make our digital twins appealing in signiﬁcant applications

such as monitoring the health of critical systems and forecasting their potential collapse induced by

environmental changes.

I. INTRODUCTION

The concept of digital twins originated from aerospace

engineering for aircraft structural life prediction [1]. In

general, a digital twin can be used for predicting dynam-

ical systems and generating solutions of emergent behav-

iors that can potentially be catastrophic [2]. Digital twins

have attracted a great deal of attention from a wide range

of ﬁelds [3] including medicine and health care [4, 5].

For example, the idea of developing medical digital twins

in viral infection through a combination of mechanistic

knowledge, observational data, medical histories, and ar-

tiﬁcial intelligence has been proposed recently [6], which

can potentially lead to a powerful addition to the exist-

ing tools to combat future pandemics. In a more dra-

matic development, the European Union plans to fund

the development of digital twins of Earth for its green

transition [7, 8].

The physical world is nonlinear. Many engineering sys-

tems, such as complex infrastructural systems, are gov-

erned by nonlinear dynamical rules, too. In nonlinear dy-

namics, various bifurcations leading to chaos and system

collapse can take place [9]. For example, in ecology, en-

vironmental deterioration caused by global warming can

lead to slow parameter drift towards chaos and species

extinction [10, 11]. In an electrical power system, volt-

age collapse can occur after a parameter shift that lands

the system in transient chaos [12]. The various climate

systems in diﬀerent geographic regions of the world are

also nonlinear and the emergent catastrophic behaviors

as the result of increasing human activities are of grave

concern. In all these cases, it is of interest to develop

∗Ying-Cheng.Lai@asu.edu

a digital twin of the system of interest to monitor its

“health” in real time as well as for predictive problem

solving in the sense that, if the digital twin indicates

a possible system collapse in the future, proper control

strategies should and can be devised and executed in time

to prevent the collapse.

What does it take to create a digital twin for a non-

linear dynamical system? For natural and engineering

systems, there are two general approaches: one is based

on mechanistic knowledge and another is based on ob-

servational data. In principle, if the detailed physics of

the system is well understood, it should be possible to

construct a digital twin through mathematical modeling.

However, there are two diﬃculties associated with this

modeling approach. First, a real-world system can be

high-dimensional and complex, preventing the rules gov-

erning its dynamical evolution from being known at a

suﬃciently detailed level. Second, the hallmark of chaos

is sensitive dependence on initial conditions. Because

no mathematical model of the underlying physical sys-

tem can be perfect, the small deviations and high di-

mensionality of the system coupled with environmental

disturbances can cause the model predictions of the fu-

ture state of the system to be inaccurate and completely

irrelevant [13, 14]. These diﬃculties motivate the propo-

sition that data-based approach can have advantages in

many realistic scenarios and a viable method to develop

a digital twin is through data. While in certain cases,

approximate system equations can be found from data

through sparse optimization [15–17], the same diﬃculties

with the modeling approach arise. These considerations

have led us to exploit machine learning to create digital

twins for nonlinear dynamical systems.

Given a nonlinear dynamical system, its digital twin is

also a dynamical system, rendering appropriate exploita-

tion of recurrent neural networks that can be designed

arXiv:2210.06144v1 [nlin.AO] 5 Oct 2022

to generate self-dynamical evolution with memory. In

this regard, reservoir computers (RC) [18–20] that have

been extensively studied in recent years [21–43] provide a

starting point, which can be trained from observational

data to generate closed-loop dynamical evolution that

follows the evolution of the target system for a ﬁnite

amount of time. Another advantage of RC is that no

back-propagation is needed for optimizing the parame-

ters - only a linear regression is required in the training

so it is computationally eﬃcient. A common situation is

that the target system is subject to external driving, such

as a driven laser, a regional climate system, or an ecosys-

tem under external environmental disturbances. Accord-

ingly, the digital twin must accommodate a mechanism

to control or steer the dynamics of the RC neural net-

work to account for the external driving. Introducing a

control mechanism into the RC structure with an exoge-

nous control signal acting directly onto the RC network

distinguishes our work from existing ones in the litera-

ture of RC as applied to nonlinear dynamical systems.

Of particular interest is whether the collapse of the tar-

get chaotic system can be anticipated from the digital

twin. The purpose of this paper is to demonstrate that

the digital twin so created can accurately produce the

bifurcation diagram of the target system and faithfully

mimic its dynamical evolution from a statistical point of

view. The digital twin can then be used to monitor the

present and future “health” of the system. More impor-

tantly, with proper training from observational data the

twin can reliably anticipate system collapses, providing

early warnings of potentially catastrophic failures of the

system.

More speciﬁcally, using three prototypical systems

from optics, ecology, and climate, respectively, we

demonstrate that the RC based digital twins developed

in this paper solve the following challenging problems:

(1) extrapolation of the dynamical evolution of the target

system into certain “uncharted territories” in the param-

eter space, (2) long-term continual forecasting of nonlin-

ear dynamical systems subject to non-stationary external

driving with sparse state updates, (3) inference of hidden

variables in the system and accurate prediction of their

dynamical evolution into the future, (4) adaptation to

external driving of diﬀerent waveform, and (5) extrapo-

lation of the global bifurcation behaviors of network sys-

tems to some diﬀerent sizes. These features make our

digital twins appealing in applications.

II. METHODS

The basic construction of the digital twin of a nonlin-

ear dynamical system [45] is illustrated in Fig. 1. It is

essentially a recurrent RC neural network with a control

mechanism, which requires two types of input signals:

the observational time series for training and the con-

trol signal f(t) that remains in both the training and

self-evolving phase. The hidden layer hosts a random or

complex network of artiﬁcial neurons. During the train-

ing, the hidden recurrent layer is driven by both the in-

put signal u(t) and the control signal f(t). The neurons

in the hidden layer generate a high-dimensional nonlin-

ear response signal. Linearly combining all the responses

of these hidden neurons with a set of trainable and opti-

mizable parameters yields the output signal. Speciﬁcally,

the digital twin consists of four components: (i) an input

subsystem that maps the low-dimensional (Din) input

signal into a (high) Dr-dimensional signal through the

weighted Dr×Din matrix Win, (ii) a reservoir network

of Nneurons characterized by Wr, a weighted network

matrix of dimension Dr×Dr, where DrDin, (iii) an

readout subsystem that converts the Dr-dimensional sig-

nal from the reservoir network into an Dout-dimensional

signal through the output weighted matrix Wout, and (iv)

a controller with the matrix Wc. The matrix Wrdeﬁnes

the structure of the reservoir neural network in the hid-

den layer, where the dynamics of each node are described

by an internal state and a nonlinear hyperbolic tangent

activation function.

The matrices Win,Wc, and Wrare generated ran-

domly prior to training, whereas all elements of Wout

are to be determined through training. Speciﬁcally,

the state updating equations for the training and self-

evolving phases are, respectively,

r(t+∆t) = (1 −α)r(t)

+αtanh [Wrr(t) + Winu(t) + Wcf(t)],(1)

r(t+∆t) = (1 −α)r(t)

+αtanh [Wrr(t) + WinWoutr0(t) + Wcf(t)],(2)

where r(t) is the hidden state, u(t) is the vector of input

training data, ∆tis the time step, the vector tanh (p)

is deﬁned to be [tanh (p1),tanh (p2), . . .]Tfor a vector

p= [p1, p2, ...]T, and αis the leakage factor. During

the training, several trials of data are typically used un-

der diﬀerent driving signals so that the digital twin can

“sense, learn, and mingle” the responses of the target sys-

tem to gain the ability to extrapolate a response to a new

driving signal that has never been encountered before.

We input these trials of training data, i.e., a few pairs of

u(t) and the associated f(t), through the matrices Win

and Wcsequentially. Then we record the state vector r(t)

of the neural network during the entire training phase as

a matrix R. We also record all the desired output, which

is the one-step prediction result v(t) = u(t+ ∆t), as

the matrix V. To make the readout nonlinear and to

avoid unnecessary symmetries in the system [24, 46], we

change the matrix Rinto R0by squaring the entries of

even dimensions in the states of the hidden layer. [The

vector (r0(t) in Eq. (2) is deﬁned in a similar way.] We

carry out a linear regression between Vand R0, with a

`-2 regularization coeﬃcient β, to determine the readout

matrix:

Wout =V · R0T(R0· R0T+βI)−1.(3)

To achieve acceptable learning performance, optimiza-

tion of hyperparameters is necessary. The four widely

v(t)

𝒲in r(t) 𝒲out

Input layer

Hidden layer

Output layer

𝒲r

u(t)

Controller 𝑓(𝑡)

Closed loop operation:

a self-evolving

dynamical system

during predicting

Open loop operation

for training

FIG. 1. Basic structure of the digital twin of a chaotic system. It consists of three layers: the input layer, the hidden recurrent

layer, an output layer, as well as a controller component. The input matrix Win maps the Din-dimensional input chaotic data to

a vector of much higher dimension Dr, where DrDin. The recurrent hidden layer is characterized by the Dr×Drweighted

matrix Wr. The dynamical state of the ith neuron in the reservoir is ri, for i= 1,...,Dr. The hidden-layer state vector is r(t),

which is an embedding of the input [44]. The output matrix Wout readout the hidden state into the Dout-dimensional output

vector. The controller provides an external driving signal f(t) to the neural network. During training, the vector u(t) is the

input data, and the blue arrow exists during the training phase only. In the predicting phase, the output vector v(t) is directly

fed back to the input layer, generating a closed-loop, self-evolving dynamical system, as indicated by the red arrow connecting

v(t) to u(t). The controller remains on in both the training and predicting phases.

used global optimization methods are genetic algo-

rithm [47–49], particle swarm optimization [50, 51],

Bayesian optimization [52, 53], and surrogate optimiza-

tion [54–56]. We use the surrogate optimization (the al-

gorithm surrogateopt in Matlab). The hyperparameters

that are optimized include d- the average degree of the

recurrent network in the hidden layer, λ- the spectral

radius of the recurrent network, kin - the scaling factor

of Win,kc- the scaling of Wc,c0- the bias in Eq. (1) and

(2), α- the leakage factor, and β- the `-2 regularization

coeﬃcient. In this paper, the validation of the RC net-

works are done with the same driving signals f(t) as in

the training data. We test driving signals f(t) that are

diﬀerent from those generating the training data (e.g.,

with diﬀerent amplitude, frequency, or waveform). To

generate the predicted bifurcation diagrams, we let the

RC networks make predictions for long enough periods to

approach the asymptotic behavior. During the warming-

up process to initialize the RC networks prior to making

the predictions, we feed randomly chosen short segments

of the training time series to feed into the RC network.

That is, no data from the target system under the testing

driving signals f(t) are required for making the predic-

tions.

III. RESULTS

For clarity, we present results on the digital twin for

a prototypical nonlinear dynamical systems with ad-

justable phase-space dimension: the Lorenz-96 climate

network model [57]. In the appendix, we present two ad-

ditional examples: a chaotic laser (Appendix A) and a

driven ecological system (Appendix B), together with a

number of pertinent issues.

A. A low-dimensional Lorenz-96 climate network

and its digital twin

The Lorenz-96 system [57] is an idealized atmospheric

climate model. Mathematically, the toy climate system is

described by mcoupled ﬁrst-order nonlinear diﬀerential

equations subject to external periodic driving f(t):

dxi

dt =xi−1(xi+1 −xi−2)−xi+f(t),(4)

where i= 1, . . . , m, is the spatial index. Under the peri-

odic boundary condition, the mnodes constitute a ring

network, where each node is coupled to three neighboring

nodes. To be concrete, we set m= 6 (more complex high-

dimensional cases are treated below). The driving force

is sinusoidal with a bias F:f(t) = Asin(ωt) + F. We ﬁx

ω= 2 and F= 2, and use the forcing amplitude Aas

Driving the

real system

Driving the

real system

Driving the

digital twin

Driving the

digital twin

FIG. 2. Digital twin of the Lorenz-96 climate system. The toy climate system is described by six coupled ﬁrst-order nonlinear

diﬀerential equations (phase-space dimension m= 6), which is driven by a sinusoidal signal f(t) = Asin(ωt) + F. (A1,A2)

Ground truth: chaotic and quasi-periodic dynamics in the system for A= 2.2 and A= 1.6, respectively, for ω= 2 and F= 2.

The sinusoidal driving signals f(t) are schematically illustrated. (B1, B2) The corresponding dynamics of the digital twin under

the same driving signal f(t). Training of the digital twin is conducted using time series from the chaotic regime. The result

in (B2) indicates that the digital twin is able to extrapolate outside the chaotic regime to generate the unseen quasi-periodic

behavior. (C, D) True and digital-twin generated bifurcation diagrams of the toy climate system, where the four vertical red

dashed lines indicate the values of driving amplitudes A, from which the training time series data are obtained. The remarkable

agreement between the two bifurcation diagrams attests to the strong ability of the digital twin to reproduce the distinct

dynamical behaviors of the target climate system in diﬀerent parameter regimes, even with training data only in the chaotic

regime. Note that there are mismatches in the details such as the positions of some periodic windows.

the bifurcation parameter. For relatively large values of

A, the system exhibits chaotic behaviors, as exempliﬁed

in Fig. 2(A1) for A= 2.2. Quasi-periodic dynamics arise

for smaller values of A, as exempliﬁed in Fig. 2(A2). As

Adecreases from a large value, a critical transition from

chaos to quasi-periodicity occurs at Ac≈1.9. We train

the digital twin with time series from four values of A, all

in the chaotic regime: A= 2.2,2.6,3.0,and 3.4. The size

of the random reservoir network is Dr= 1,200. For each

value of Ain the training set, the training and validation

lengths are t= 2,500 and t= 12, respectively, where

the latter corresponds to approximately ﬁve Lyapunov

times. The warming-up length is t= 20 and the time

step of the reservoir dynamical evolution is ∆t= 0.025.

The hyperparameter values (See Sec. II for their mean-

ings) are optimized to be d= 843, λ= 0.48, kin = 0.29,

kc= 0.113, α= 0.41, and β= 1 ×10−10. Our compu-

tations reveal that, for the deterministic version of the

Lorenz-96 model, it is diﬃcult to reduce the validation

error below a small threshold. However, adding an appro-

priate amount of noise into the training time series [18]

can lead to smaller validation errors. We add an additive

Gaussian noise with standard deviation σnoise to each

input data channel to the reservoir network [including

the driving channel f(t)]. The noise amplitude σnoise is

treated as an additional hyperparameter to be optimized.

For the toy climate system, we test several noise levels

and ﬁnd the optimal noise level giving the best validating

performance: σnoise ≈10−3.

Figures 2(B1) and 2(B2) show the dynamical behav-

iors generated by the digital twin for the same values

of Aas in Figs. 2(A1) and 2(A2), respectively. It can

be seen that not only does the digital twin produce the

correct dynamical behavior in the same chaotic regime

where the training is carried out, it can also extrapolate

beyond the training parameter regime to correctly pre-

dict the unseen system dynamics there (quasiperiodicity

in this case). To provide support in a broader parameter

range, we calculate true bifurcation diagram, as shown in

Fig. 2(C), where the four vertical dashed lines indicate

the four values of the training parameter. The bifurca-

tion generated by the digital twin is shown in Fig. 2(D),

which agrees remarkably well with the true diagram even

at a detailed level. Note that there are mismatches in the

details such as the positions of some periodic windows

in Figs. 2(C) and 2(D). To predict all the features in a

bifurcation diagram requires extensive interpolation and

extrapolation of the system dynamics in the phase space.

Previously, it was suggested that RC can have a cer-

tain degree of extrapolability [34–39]. Figure 2 repre-

sents an example where the target system’s response is

extrapolated to external sinusoidal driving with unseen

amplitudes. In general, extrapolation is a diﬃcult prob-

lem. Some limitations of the extrapolability with respect

to the external driving signal is discussed in Appendix

A, where the digital twin can predict the crisis point but

cannot extrapolate the asymptotic behavior after the cri-

sis.

In the following, we systematically study the applica-

bility of the digital twin in solving forecasting problems

in more complicated situations than the basic settings

demonstrated in Fig. 2. The issues to be addressed are

high dimensionality, the eﬀect of the waveform of the

driving on forecasting, and the generalizability across

Lorenz-96 networks of diﬀerent sizes. Results of contin-

ual forecasting and inferring hidden dynamical variables

using only rare updates of the observable are presented

in Appendices C and D, respectively.

B. Digital twins of parallel RC neural networks for

high-dimensional Lorenz-96 climate networks

We extend the methodology of digital twin to high-

dimensional Lorenz-96 climate networks, e.g., m= 20.

To deal with such a high-dimensional target system, if a

single reservoir system is used, the required size of the

neural network in the hidden layer will be too large to

be computationally eﬃcient. We thus turn to the par-

allel conﬁguration [25] that consists of many small-size

RC networks, each “responsible” for a small part of the

target system. For the Lorenz-96 network with m= 20

coupled nodes, our digital twin consists of ten parallel RC

networks, each monitoring and forecasting the dynamical

evolution of two nodes (Dout = 2). Because each node in

the Lorenz-96 network is coupled to three nearby nodes,

we set Din =Dout +Dcouple = 2 + 3 = 5 to ensure that

suﬃcient information is supplied to each RC network.

The speciﬁc parameters of the digital twin are as fol-

lows. The size of the recurrent layer is Dr= 1,200. For

each training value of the forcing amplitude A, the train-

ing and validation lengths are t= 3,500 and t= 100,

respectively. The “warming up” length is t= 20 and the

time step of the dynamical evolution of the digital twin

is ∆t= 0.025. The optimized hyperparameter values

are d= 31, λ= 0.75, kin = 0.16, kc= 0.16, α= 0.33,

β= 1 ×10−12, and σnoise = 10−2.

The periodic signal used to drive the Lorenz-96 cli-

mate network of 20 nodes is f(t) = Asin(ωt) + Fwith

ω= 2, and F= 2. The structure of the digital twin con-

sists of 20 small RC networks as illustrated in Fig. 3(A).

Figures 3(B1) and 3(B2) show a chaotic and a periodic

attractor for A= 1.8 and A= 1.6, respectively, in the

(x1, x2) plane. Training of the digital twin is conducted

by using four time series from four diﬀerent values of

A, all in the chaotic regime. The attractors generated

by the digital twin for A= 1.8 and A= 1.6 are shown

in Figs. 3(C1) and 3(C2), respectively, which agree well

with the ground truth. Figure 3(D) shows the bifurca-

tion diagram of the target system (the ground truth),

where the four values of A:A= 1.8, 2.2, 2.6, and 3.0,

from which the training chaotic time series are obtained,

are indicated by the four respective vertical dashed lines.

The bifurcation diagram generated by the digital twin is

shown in Fig. 3(E), which agrees well with the ground

truth in Fig. 3(D).

C. Digital twins under external driving with varied

waveform

The external driving signal is an essential ingredient

in our articulation of the digital twin, which is particu-

larly relevant to critical systems of interest such as the

climate systems. In applications, the mathematical form

of the driving signal may change with time. Can a dig-

ital twin produce the correct system behavior under a

driving signal that is diﬀerent than the one it has “seen”

during the training phase? Note that, in the examples

treated so far, it has been demonstrated that our digital

twin can extrapolate the dynamical behavior of a target

system under a driving signal of the same mathematical

form but with a diﬀerent amplitude. Here, the task is

more challenging as the form of the driving signal has

changed.

As a concrete example, we consider the Lorenz-96 cli-

mate network of m= 6 nodes, where a digital twin is

trained with a purely sinusoidal signal f(t) = Asin(ωt)+

F, as illustrated in the left column of Fig. 4(A). During

the testing phase, the driving signal has the form of the

sum of two sinusoidal signals with diﬀerent frequencies:

f(t) = A1sin(ω1t) + A2sin(ω2t+ ∆φ) + F, as illustrated

in the right panel of Fig. 4(A). We set A1= 2, A2= 1,

ω1= 2, ω2= 1, F= 2, and use ∆φas the bifurca-

tion parameter. The RC parameter setting is the same

as that in Fig. 2. The training and validating lengths

for each driving amplitude Avalue are t= 3,000 and

t= 12, respectively. We ﬁne that this setting prevents

the digital twin from generating an accurate bifurcation

diagram, but a small amount of dynamical noise to the

target system can improve the performance of the digital

twin. To demonstrate this, we apply an additive noise

term to the driving signal f(t) in the training phase:

df(t)/dt =ωA cos(ωt) + δDNξ(t), where ξ(t) is a Gaus-

sian white noise of zero mean and unit variance, and δDN

is the noise amplitude (e.g., δDN = 3 ×10−3). We use

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DigitaltwinsofnonlineardynamicalsystemsLing-WeiKong,1YangWeng,1BryanGlaz,2MulugetaHaile,2andYing-ChengLai1,3,1SchoolofElectrical,ComputerandEnergyEngineering,ArizonaStateUniversity,Tempe,Arizona85287,USA2VehicleTechnologyDirectorate,CCDCArmyResearchLaboratory,2800PowderMillRoad,Adelphi,MD20783-1138...

展开>> 收起<<

Digital twins of nonlinear dynamical systems Ling-Wei Kong1Yang Weng1Bryan Glaz2Mulugeta Haile2and Ying-Cheng Lai1 3 1School of Electrical Computer and Energy Engineering.pdf

共21页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Digital twins of nonlinear dynamical systems Ling-Wei Kong1Yang Weng1Bryan Glaz2Mulugeta Haile2and Ying-Cheng Lai1 3 1School of Electrical Computer and Energy Engineering

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: