Characterizing information loss in a chaotic double pendulum with the Information Bottleneck Kieran A. Murphy1and Dani S. Bassett1234567

2025-04-30 0 0 1.86MB 7 页 10玖币

侵权投诉

Characterizing information loss in a chaotic double

pendulum with the Information Bottleneck

Kieran A. Murphy1and Dani S. Bassett1,2,3,4,5,6,7

1Dept. of Bioengineering, School of Engineering & Applied Science,

U. of Pennsylvania, Philadelphia, PA 19104, USA

2Dept. of Electrical & Systems Engineering, School of Engineering & Applied Science,

U. of Pennsylvania, Philadelphia, PA 19104, USA

3Dept. of Neurology, Perelman School of Medicine, U. of Pennsylvania, Philadelphia, PA 19104, USA

4Dept. of Psychiatry, Perelman School of Medicine, U. of Pennsylvania, Philadelphia, PA 19104, USA

5Dept. of Physics & Astronomy, College of Arts & Sciences, U. of Pennsylvania, Philadelphia, PA 19104, USA

6The Santa Fe Institute, Santa Fe, NM 87501, USA

7To whom correspondence should be addressed: dsb@seas.upenn.edu

Abstract

A hallmark of chaotic dynamics is the loss of information with time. Although

information loss is often expressed through a connection to Lyapunov exponents—

valid in the limit of high information about the system state—this picture misses the

rich spectrum of information decay across different levels of granularity. Here we

show how machine learning presents new opportunities for the study of information

loss in chaotic dynamics, with a double pendulum serving as a model system. We

use the Information Bottleneck as a training objective for a neural network to

extract information from the state of the system that is optimally predictive of the

future state after a prescribed time horizon. We then decompose the optimally

predictive information by distributing a bottleneck to each state variable, recovering

the relative importance of the variables in determining future evolution. The

framework we develop is broadly applicable to chaotic systems and pragmatic to

apply, leveraging data and machine learning to monitor the limits of predictability

and map out the loss of information.

1 Introduction

A fundamental aspect of chaos is the loss of information over time: for any measurement of a chaotic

system with ﬁnite resolution, there is a ﬁnite time horizon beyond which the measurement bears

no predictive power [

]. The premise of this work is simple: to ﬁnd the optimally

predictive information in a chaotic system at different levels of granularity, and to study how the

predictive power of this information erodes with the passage of time.

Information loss is intimately connected to the distortion of regions in phase space by chaotic

dynamics, and thus to Lyapunov exponents: the sum of the positive Lyapunov exponents gives the

Kolmogorov-Sinai (KS) entropy, the average rate of information loss[

]. However, these quantities

are valid in the limit of maximal information—where inﬁnitesimally-separated trajectories are

discernible—and are thus somewhat removed from reality [

]. The

(, τ)

-entropy generalizes the KS

entropy, describing the loss of predictive power for different amounts of information about the system

state [

]. It is deﬁned by way of a rate-distortion objective that minimizes the rate of information

needed to predict the system state better than a threshold value of some chosen measure of distortion.

The Information Bottleneck (IB) is a rate-distortion problem where the measure of distortion is based

on mutual information; it extracts the information from one variable that is most shared with a second

variable [

]. We can use the IB to ﬁnd optimally predictive information from one state of a chaotic

system about a future state, and at the same time measure the loss of predictive power [

]. In this work

we develop a framework that uses the IB for analyzing chaotic dynamics with machine learning. We

arXiv:2210.14220v1 [cs.LG] 25 Oct 2022

a cb d e

Figure 1:

Predicting the future of a double pendulum given ﬁnite information. (a)

The double

pendulum system with four state variables

θ1,˙

θ1, θ2,˙

θ2

(b)

At time

the state

is measured,

yielding a random variable

. The ﬁnite mutual information between the state and the measurement

I(St;Ut)

manifests as a continuum of underlying states that cannot be discerned given an outcome

(c)

Information is lost over time due to the double pendulum’s chaotic dynamics. The

representation

contains less information about the future state

St+∆

than it does about the present

state.

(d)

The measurement

is a function of

and can be learned with a neural network. To

extract the information from the current state Stmost predictive of the future state, we optimize the

Information Bottleneck objective.

(e)

By distributing bottlenecks, we gain marked interpretability by

monitoring the share of optimally predictive information across state variables.

use an interpretable variant of the IB, the Distributed IB [

], to decompose the optimally predictive

information in terms of a system’s state variables.

2 Approach

Our testbed is a double pendulum (Fig. 1a): one of the simplest physical systems to exhibit chaotic

behavior [23]. We simulated 10,000 trajectories at constant energy (details in the Appendix).

Given a random variable

St∼p(st)

for the system state at time

, a measurement

is any (possibly

stochastic) function of the system state:

Ut∼p(ut|st)

. A measurement process shares mutual

information

I(Ut;St)

with the underlying state, deﬁned as the reduction in Shannon’s entropy [

]

about

once

is known. For a continuous variable

and without inﬁnite measurement capabilities,

the outcome of

can only narrow down the possible values of

, which we visualize in Fig. 1b as a

region of possible pendulum states. We can similarly examine the reduction in entropy about a future

state after time

∆

has elapsed, given the same measurement

. Then

I(Ut;St+∆)

serves as an upper

bound of the predictive capabilities of any forecasting device given the outcome of the measurement

Ut. The passage of time invariably expands our uncertainty about the system state (Fig. 1c).

Importantly, different measurements of the state—that is, different measurement processes

, not

different outcomes

of the same measurement—can have identical

I(Ut;St)

but vary in the in-

formation shared with the future state. We seek the measurement

that is most predictive of the

future dynamics, for a given allowance of information about the present. To ﬁnd this optimally

predictive information, we use the Information Bottleneck [

] and optimize over the space of

possible measurements. The following objective maximizes information about the future state and

the information about the current state:

LIB =βI(Ut;St)−I(Ut;St+∆).(1)

The parameter

controls the bottleneck, restricting the information preserved about the present state.

As the bottleneck strength

varies, a trajectory in the “information plane” [

] is traced (Fig. 2a),

which shows the exchange rate of information preserved about the present state versus information

predictive of the future state, for different time horizons

∆

can have no more information about

St+∆

than it does about

, so no trajectory can exist above the line with slope 1. Optimal

are as

close to this line as possible, and for a chaotic system longer time horizons are further displaced.

Measuring mutual information from data is notoriously tricky [

]. To be compatible with

machine learning, the Variational Information Bottleneck (VIB) [

] replaces the mutual information

terms of Eqn. 1with bounds in a framework nearly identical to that of Variational Autoencoders [

10]. The bottleneck of Eqn. 1is replaced with an upper bound on the mutual information,

I(Ut;St)≤DKL(p(ut|st)||r(u)).(2)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

CharacterizinginformationlossinachaoticdoublependulumwiththeInformationBottleneckKieranA.Murphy1andDaniS.Bassett1,2,3,4,5,6,71Dept.ofBioengineering,SchoolofEngineering&AppliedScience,U.ofPennsylvania,Philadelphia,PA19104,USA2Dept.ofElectrical&SystemsEngineering,SchoolofEngineering&AppliedScience,U.o...

展开>> 收起<<

Characterizing information loss in a chaotic double pendulum with the Information Bottleneck Kieran A. Murphy1and Dani S. Bassett1234567.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Characterizing information loss in a chaotic double pendulum with the Information Bottleneck Kieran A. Murphy1and Dani S. Bassett1234567

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: