Quantifying Complexity An Object-Relations Approach to Complex Systems Stephen Casey1

2025-05-02 0 0 2.48MB 13 页 10玖币

侵权投诉

Quantifying Complexity:

An Object-Relations Approach to Complex Systems

Stephen Casey1

NASA Langley Research Center

(*Electronic mail: Stephen.Casey@nasa.gov)

(Dated: 10 November 2022)

The best way to model, understand, and quantify the information contained in complex systems is an open question in

physics, mathematics, and computer science. The uncertain relationship between entropy and complexity further com-

plicates this question. With ideas drawn from the object-relations theory of psychology, this paper develops an object-

relations model of complex systems which generalizes to systems of all types, including mathematical operations,

machines, biological organisms, and social structures. The resulting Complex Information Entropy (CIE) equation is a

robust method to quantify complexity across various contexts. The paper also describes algorithms to iteratively update

and improve approximate solutions to the CIE equation, to recursively infer the composition of complex systems, and

to discover the connections among objects across different lengthscales and timescales. Applications are discussed in

the ﬁelds of engineering design, atomic and molecular physics, chemistry, materials science, neuroscience, psychology,

sociology, ecology, economics, and medicine.

How can one observe the behavior of a system and deter-

mine its inner workings? It is usually impossible to solve

this problem exactly, since most real-world models con-

tain uncertainty in relation to the systems they represent.

There is a tradeoff between model accuracy, model size,

and computational cost. There are often also hidden vari-

ables and elements of stochasticity.

I. INTRODUCTION

Minimum Description Length (MDL) is often used as a pa-

rameter in model selection to ﬁt a particular dataset, whereby

the shortest description of the data, or the model with the best

compression ratio, is assumed to be the best model1. MDL

is a mathematical implementation of Occam’s razor, which

dictates that among competing models, the model with the

fewest assumptions is to be preferred. If an MDL descrip-

tion can contain all of the operations in a Turing-complete

programming language, the description length then becomes

equivalent to the Kolmogorov complexity, which is the length

of the shortest computer program that produces the dataset as

output2. For a given dataset, a program is Pareto-optimal if

there is no shorter program that produces a more accurate out-

put. The graph of Kolmogorov program length vs. accuracy

is the Pareto frontier of a dataset.

If one combines Occam’s razor with the Epicurean Prin-

ciple of Multiple Explanations, the result is Solomonoff’s

theory of inductive inference34. According to the Principle

of Multiple Explanations, if more than one theory is consis-

tent with the observations, all such theories should be kept.

Solomonoff’s induction considers all computable theories that

may have generated an observed dataset, while assigning a

higher Bayesian posterior probability to shorter computable

theories. The theory of inductive inference uses a computa-

tional formalization of Bayesian statistics to consider multiple

competing programs simultaneously in accordance with the

Principle of Multiple Explanations, while prioritizing shorter

programs in accordance with Occam’s razor.

A characteristic feature of complex systems is their activ-

ity across a wide range of lengthscales and timescales. As

a consequence, they can often be divided into a hierarchy of

sub-components. In order to reduce the computational dif-

ﬁculty of programs that can be divided into sub-programs,

the dynamic programming method, originally developed by

Richard Bellman5, simpliﬁes a complicated problem by re-

cursively breaking it into simpler sub-problems. The divide-

and-conquer technique used by dynamic programming can

applied to both computer programming and mathematical op-

timization. A problem is said to have optimal substructure

if it can be solved optimally by recursively breaking into

sub-problems and ﬁnding the optimal solution to each sub-

problem.

In contrast to the divide-and-conquer procedure, a uniﬁ-

cation procedure ﬁnds a single underlying theory that can ex-

plain two or more separate higher-level theories. Causal learn-

ing algorithms or structural learning algorithms are examples

of uniﬁcation procedures, which determine cause-and-effect

among variables in an observed dataset. These variables may

be hidden or unknown, in which case they must be inferred via

hidden Markov models, Bayesian inference models, or other

methods.

These four principles – Occam’s razor, the Principle of

Multiple Explanations, divide-and-conquer, and uniﬁcation –

have been used in various combinations to analyze complex

systems. Some researchers have used them all together, such

as Wu, Udrescu, and Tegmark in their endeavor to create an

AI physicist789. In order to fully characterize the informa-

tion contained in complex systems, a ﬁfth item should be

added to this list: the principle of overinterpretation. This

principle states there can be multiple correct interpretations

of the same dataset, and the validity of any particular lens

of interpretation is determined by whether or not it produces

useful insights. This is distinct from the Principle of Multi-

ple Explanations, which states that no plausible theory can be

ruled out, but which still assumes there is one correct theory

arXiv:2210.12347v2 [cs.LG] 9 Nov 2022

FIG. 1: a) Objects contain qualia, which are properties, variables, or other objects. Affordances are modes of interaction. b) The relationships among objects

can be shown using system diagrams. In this example, a mechanical watch is composed of a dial object and a movement object. The movement object is

composed of gears, regulators, and springs, which affect each other through mechanical affordances. c) The categories used for biological classiﬁcation are not

exact, but are hypothetical objects inferred by scientists and subject to revision. This graph shows the categories for cetaceans between the ranks of order and

family. d) Connected objects in the graph share some amount of structure and mutual information. In this language diagram, depicting the derivatives of the

Balto-Slavic language which may have split into Baltic and Slavic around 1400 BCE6, each language derives most of its information and structure from its

predecessor in the graph. Image licensed under CC BY-SA 3.0.

among many plausible. In contrast to the Principle of Multi-

ple Explanations, the principle of overinterpretation states that

multiple interpretations can be simultaneously correct. In the

same way that the Principle of Multiple Explanations was sug-

gested by Epicurus and reﬁned by Bayes or Laplace, the prin-

ciple of overinterpretation may be said to have been suggested

by Freud in his analysis of Hamlet in The Interpretation of

Dreams10 and reﬁned by Walter Kaufmann11. In a mathemati-

cal context, overinterpretation indicates there may be multiple

non-equivalent Pareto-optimal objects within a given dataset.

II. THEORY

The deﬁnitions used here are as general as possible. The

word object is deﬁned as an entity containing two features: 1.

A set of properties, variables, or other objects, called qualia

(the singular form used here is qualium), and 2. functions

between itself and other objects, called affordances. A sum-

mary is shown in Figure 1a.

An example for a mechanical watch is shown in Figure 1b.

The watch is composed of the dial and the movement, which

are themselves composed of constituent objects. Systems en-

gineers and computer scientists will recognize the similarities

to system design documents and Uniﬁed Modeling Language

(UML) diagrams.

A. Hierarchical Graphs

In addition to representing objects using component dia-

grams or Venn diagrams, it is often useful to employ graph de-

compositions. A taxonomy diagram for cetaceans is shown in

Figure 1c, which separates general categories into more spe-

ciﬁc categories. Similarly, Figure 1d shows the evolutionary

diagram of the Balto-Slavic languages. These graphs are hi-

erarchical in the sense that each object derives much of its

information and structure from its predecessor.

B. Constructing and Deconstructing Objects

What exactly constitutes an object? Roughly, an object is

an information-theoretic construct that contains a lot of in-

formation but does not require much description. Shea12 has

applied this deﬁnition in physics by analyzing a ﬂuid system

containing both laminar and turbulent ﬂow. Without prepro-

grammed knowledge of different ﬂow regimes, his machine

learning algorithm discovered the boundary between the lam-

inar and turbulent zones by minimizing the complexity of the

model relative to its predictive capability.

In simple algebraic systems, objects can be symbolic vari-

ables while affordances are mathematical functions. Udrescu8

has employed this formulation in the development of a ma-

chine learning algorithm to discover well-known physical

laws from tables of experimental data. Figure 2 shows

this process for Newton’s law of universal gravitation F=

Gm1m2/r2. Given a table of G,m,x,yand zvalues, the pro-

gram ﬁrst discovers the dimensionless quantity Gm2

1/x2

1, then

the ratio of masses m2/m1, then ﬁnally discovers the relation-

ship among the position variables via polynomial regression.

The uncertainty present in the dataset is reduced each time a

new object is found.

1. Structural Learning Methods

Much previous work in learning the structural composition

of systems has focused on Bayesian belief networks and hid-

den Markov models. A Bayesian belief network is a proba-

bilistic model that represents a set of variables and their con-

ditional dependencies via a directed acyclic graph, meaning a

graph in which information ﬂows in only one direction along

each edge, and which do not contain any circular causation

loops. These graphs are ideal for predicting the probable

causes and contributing factors of observed events. Algo-

rithms exist for inferring unobserved variables, in addition to

algorithms for belief propagation which can conditional prob-

abilities throughout the graph in response to new evidence.

A hidden Markov model (HMM) assumes the system un-

der investigation is a Markov process, which is a network

of possible states of the system and the probabilities of tran-

sitions between states. HMM’s attempt to infer unobserved

hidden states of the system. The graphs used for Markov ran-

dom ﬁelds can be undirected and contain cycles, which are

more general than the directed acyclic graphs used for belief

nets. Consequently, HMM’s are a useful analysis tool that

have found applications across numerous ﬁelds.

While a great deal of progress has been made in structural

inference algorithms such as Bayesian belief networks and

FIG. 2: ML is used to procedurally ﬁnd objects within a table of simulated

gravitational measurements. Image adapted from Figure 2 in8. Used with

permission.

hidden Markov models, they are limited in their ability to infer

relationships among variables beyond conditional probabili-

ties. Conway’s Game of Life is a good example. The Game

of Life is a well-known cellular automaton devised by John

Conway13 which uses simple rules to produce complex and

lifelike behavior on a two-dimensional grid of square cells us-

ing discrete timesteps. The game applies the following rules

each timestep:

1. Any live cell with fewer than two live neighbors dies.

2. Any live cell with two or three live neighbors lives.

3. Any live cell with more than three live neighbors dies.

4. Any dead cell with exactly three live neighbors be-

comes a live cell.

Although these rules are simple, they will defeat a proba-

bilistic inference algorithm’s attempt to discover them. This

is because the behavior of a cell is not stochastic in relation-

ship to its neighbors, but instead is based on the application of

basic logical operations. In order to solve this system, some-

thing more akin to Wu, Udrescu, and Tegmark’s AI physicist

is needed.

2. The Role of Machine Learning

Continuing with the Game of Life example, a more ro-

bust machine learning (ML) algorithm can be devised to in-

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

QuantifyingComplexity:AnObject-RelationsApproachtoComplexSystemsStephenCasey1NASALangleyResearchCenter(*Electronicmail:Stephen.Casey@nasa.gov)(Dated:10November2022)Thebestwaytomodel,understand,andquantifytheinformationcontainedincomplexsystemsisanopenquestioninphysics,mathematics,andcomputerscience....

展开>> 收起<<

Quantifying Complexity An Object-Relations Approach to Complex Systems Stephen Casey1.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Quantifying Complexity An Object-Relations Approach to Complex Systems Stephen Casey1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: