A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS ERHAN BAYRAKTAR ASAF COHEN AND APRIL NELLIS

2025-04-27 0 0 3.52MB 27 页 10玖币

侵权投诉

A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL

SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS

ERHAN BAYRAKTAR, ASAF COHEN, AND APRIL NELLIS

Abstract. We develop a backward-in-time machine learning algorithm that uses a sequence of neural

networks to solve optimal switching problems in energy production, where electricity and fossil fuel prices

are subject to stochastic jumps. We then apply this algorithm to a variety of energy scheduling problems,

including novel high-dimensional energy production problems. Our experimental results demonstrate that

the algorithm performs with accuracy and experiences linear to sub-linear slowdowns as dimension increases,

demonstrating the value of the algorithm for solving high-dimensional switching problem.

Keywords. Deep neural networks, forward-backward systems of stochastic diﬀerential equations, optimal

switching, Monte Carlo algorithm, optimal investment in power generation, planning problems

1. Introduction

Energy production and energy markets play a large role in the modern economy and as such, it is beneﬁcial

to both producers and consumers for electricity production to be optimized. Energy producers, in particular,

desire to operate eﬃciently despite the inherent volatility of both electricity demand and the availability of

various fuels. Determining the correct operating strategy for an energy production facility therefore requires

dynamic adjustment as the underlying drivers of price and proﬁt ﬂuctuate stochastically with supply and

demand. Recent supply-chain issues in global markets have further underlined the volatility of prices and

the need for ﬂexible optimization methods which allow producers to dynamically adapt to changes in the

energy markets.

There are multiple perspectives from which to approach problems related to energy production and pricing.

In the case where a model includes only a single power generation facility, this facility is considered a price-

taker, and its production decisions have little impact on the overall ﬂow of electricity supply and demand.

The facility’s only goal is to maximize its own proﬁt, as it is not the sole electricity producer in its region.

To this end, the facility is able to alter its own production capacity in response to exogenous outside factors.

However, we can also consider a situation in which an agent oversees multiple power generation facilities,

and has the option to bring them online or remove them. Each of these facilities is fueled by one of a

selection of fuel sources, ranging from coal to solar energy. The larger scale of this operation makes this

agent a price-setter, and so investment decisions aﬀect both electricity spot prices and their own proﬁts. In

this case, penalties could also be incurred for failing to satisfy electricity demand. Our focus will be on the

former case, but our algorithm could easily be extended to other situations.

These situations can be modeled as optimal switching problems, and in our paper we present a machine

learning algorithm that is able to solve optimal switching problems of higher dimensions than previously

studied, allowing us to consider a wider selection of fuel sources than in existing literature. Such energy

production switching problems consist of a stochastic state process (such as exogenous electricity demand and

fuel prices) which drives an objective function. At discrete “switching times” a production decision is chosen

from a discrete set of possible “modes” of production (which can model factors like capacity level or fuel

type). The controller switches between modes based on the current value of the state variable, but must pay

a penalty for such switches (usually monetary, reﬂecting resource redirection). The class of optimal switching

problems is one that has both been investigated from an analytical perspective [22, 5, 13, 14] and applied to

ﬁelds from ﬁnance [29] to cloud computing [17], but these problems remain diﬃcult to solve numerically in

higher dimensions. In the realm of energy markets, mathematicians have used optimal switching to model

power plant scheduling [12, 35], electricity spot prices [1], and run-of-river hydroelectric power generation

TO APPEAR IN SIAM JOURNAL ON FINANCIAL MATHEMATICS.

arXiv:2210.03045v2 [math.OC] 16 Sep 2023

2 ERHAN BAYRAKTAR, ASAF COHEN, AND APRIL NELLIS

[33]. Energy storage problems [18, 32, 39] are another popular application of optimal switching, but we focus

on scheduling and production problems in our current work.

Various approaches have been taken to avoid a grid-based method, as grids are very susceptible to the

so-called “curse of dimensionality”, including many Monte Carlo-based methods like [40, 1]. However, such

probabilistic approaches are also limited in the dimension they can handle, as most rely on regression

over a number of basis functions that grows quickly with the dimension of the state space. In recent

years, the applications of machine learning to mathematical problems has become more and more common,

many inspired by seminal works such as [24], which trains a neural network to minimize the global error

associated with the backward stochastic diﬀerential equation (BSDE) representation of certain classes of

partial diﬀerential equations (PDEs). Expanding upon this work, neural networks have been found to

accurately estimate the solutions of a variety of partial diﬀerential equations of varying complexities when

used in diﬀerent conﬁgurations, as in [26, 36, 6, 20]. In addition, the early paper [2] utilized neural networks

to solve for an optimal gas consumption strategy under uncertainty. It follows that such neural network-

based methods can be extended to solve optimal switching problems. Our algorithm draws upon the neural-

network-based deep backward dynamic programming approach introduced in [26] and extends it to situations

where the reﬂection boundary is no longer a known function, like the payoﬀ of an American option. Instead,

the reﬂection boundary becomes dependent on the optimal control decision at the given point in time. We

also introduce jumps in the state process, which change the associated formulation from a partial diﬀerential

equation to a partial integro-diﬀerential equation (PIDE). These jumps are incorporated into the model to

better simulate the volatility inherent in electricity and fossil fuel markets. The recent work [21] extends

[24] to a setting with jumps, and [19] applies neural networks to PIDEs that arise in insurance mathematics.

In this paper, we extend [26] to handle both jumps and switches in a wider range of problems. This

algorithm is able to handle high-dimensional problems well because the time needed for artiﬁcial neural

network computations grows only linearly in the dimension of the state variable and suﬀers only minimal

slowdowns as the dimension increases, as demonstrated in Section 4. Our code can be found at https:

//github.com/april-nellis/osj.

In Section 2, we introduce the general stochastic model of an optimal switching problem. In Section 3,

we provide some background on neural networks and detail the proposed machine learning algorithm. In

Section 4 we discuss numerical examples of energy scheduling and capacity investment, and demonstrate the

high-dimensional abilities of our algorithm1. In Section 5, we verify the convergence of the neural networks

in our proposed algorithm to the true value functions.

2. Stochastic Model

2.1. Setup. The goal of our paper is to numerically solve high-dimensional optimal switching problems

related to energy production. Consider a ﬁltered probability space (Ω,F,{Ft}t,P) satisfying the usual

conditions and supporting a d-dimensional Wiener process Wand a one-dimensional Poisson random measure

N(de, ds) with intensity measure ν(de)ds, where RRdν(de) = λ≥0. Consider further a d-dimensional jump-

diﬀusion process, given by

(2.1) Xt=x0+Zt

b(Xs)ds +Zt

σ(Xs)dWs+Zt

0ZRd

β(Xs−, e)N(de, ds), t ∈[0, T ], x0∈Rd.

Here, b:Rd→Rd,σ:Rd→Rd×d, and β:Rd×E→Rd, where dis a relatively large dimension and

E⊆Rd.

Assumption 1. We assume that

(1) The functions b,σ, and βare Lipschitz, and βis a measurable map such that there exists K > 0for

which

sup

ξ∈E|β(0, ξ)| ≤ Kand sup

ξ∈E|β(x, ξ)−β(x′, ξ)| ≤ K|x−x′|,∀x, x′∈Rd.

(2) The function β(x, ξ)has Jacobian such that ∇β(x, ξ) + Idis invertible with a bounded inverse.

Remark 1. From the appendices of [8], the conditions in assumption 1 imply the existence of a unique

adapted solution Xtto eq. (2.1).

1All calculations in this paper were performed on a 10-core CPU, 16-core GPU 2021 Macbook Pro with M1 Pro chip, without

using GPU acceleration.

A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS2

This stochastic process drives an optimal switching problem which we will solve using a series of artiﬁcial

neural networks. Variations on this problem have been studied in previous theoretical papers such as [23]

and [16], and can be summarized as trying to ﬁnd the optimal choice of control a={(τk, αk)}k∈N, where

αk∈I=: {1, . . . , I}is the regime/mode which is selected at switching time τk, such that αkis Fτk-

measurable, for any k∈N. We set τ0= 0 to denote that a switch is allowed as soon as the process begins.

The initial mode of the system, i, is therefore denoted by α−1=i. The control process (as)s∈[0,T ]associated

with ais denoted by:

as=X

αk1{τk≤s<τk+1}.

It represents the current mode of the system and the set of such strategies is given by A. The set of

admissible strategies is deﬁned as all strategies ﬁtting the above description which contain only a ﬁnite,

though potentially random, number of switches, and is denoted A. The set of admissible strategies that

begin in mode iat initial time tis denoted At,i. The expected payoﬀ function associated with the control

a∈ At,i is given by

(2.2) J(t, x, i, a) := E

ZT

fas(s, Xs)ds +gaT(XT)−X

k∈N\{0}

Cαk−1,αk(Xτk)1{t≤τk<T }Xt=x, at=i

,

where fi:R×Rd→Ris the running proﬁt in mode i,gi:Rd→Ris the terminal proﬁt if ending in mode

i, and Ci,j :Rd→Ris the cost of switching modes from ito jfor a given value of the state variable, where

i, j ∈I. Here and in the sequel, 1Bis the indicator of the event B, such that 1B(ω) = 1 if ω∈Band 0

otherwise. We can deﬁne the initial status of the system as F0:= {X0=x0, α−1=i}. All expectations are

conditioned on F0when not otherwise speciﬁed.

Assumption 2. To discourage an optimal strategy with multiple instantaneous switches, we make the fol-

lowing assumptions on the switching costs. There exists ϵ > 0such that

Ci,j (x)≥ϵ > 0,∀i, j ∈I,∀x∈Rd,

Cii(x)≡0,∀i∈I,

Ci,j (x) + Cj,k(x)≥Ci,k(x),∀i, j, k ∈I,∀x∈Rd.

These assumptions are standard in optimal switching problems, encoding “direct” switches between states,

and are enforced throughout the paper. We also make the Lipschitz assumption that there exists a constant

[C]lsuch that

|Ci,j (x1)−Ci,j (x2)| ≤ [C]l||x1−x2||,

for all x1, x2∈Rdand all i, j ∈I.

We also make certain assumptions on the running proﬁt and terminal proﬁt functions throughout the

paper.

Assumption 3. .

(1) There exists a constant [f]lsuch that for every t1, t2∈[0, T ]and x1, x2∈Rd,

|fi(t1, x1)−fi(t2, x2)| ≤ [f]l(|t1−t2|1/2+||x1−x2||),∀i∈I.

(2) We assume maxi∈Isup0≤t≤T|fi(t, 0)|<∞and fiis square-integrable on [0, T ]for all iin I.

(3) The functions {gi}i∈Iare Lipschitz continuous and satisfy linear growth conditions.

The value function is given by

V(t, x, i) := sup

a∈At,i

J(t, x, i, a).

It is a standard result in control theory (see [15, 7]) that the solution to this optimization problem can be

represented as a real-valued stochastic process Yi

t=V(t, Xt, i) that solves the stochastic diﬀerential equation

4 ERHAN BAYRAKTAR, ASAF COHEN, AND APRIL NELLIS

t=gi(XT) + ZT

fi(s, Xs)ds −ZT

(Zi

s)TdWs−ZT

tZRd

∆Yi

s(e)˜

N(de, ds) + Ri

T−Ri

t≥max

j̸=i{−Ci,j (Xt) + Yj

t}, t ∈[0, T ],

(Yi

t−max

j̸=i(−Ci,j (Xt) + Yj

t))dRi

t= 0.

(2.3)

where ˜

N(de, ds) := N(de, ds)−ν(de)ds and the reﬂection boundary Ri

tis a nondecreasing process with

0= 0. Further, the auxiliary processes Zi

tand ∆Yi

t(e) can be deﬁned as

t:= σT(Xt)Vx(t, Xt, i)∈Rd,

∆Yi

t(e) := Vt, Xt−+β(Xt−, e), i−V(t, Xt−, i)∈R.

The stochastic diﬀerential equations for Xtand Ytcomprise a system of forward-backward stochastic

diﬀerential equations (FBSDEs). In addition, the continuation values associated with beginning in mode i

at time t1and remaining in that mode over the interval [t1, t2] for t1, t2∈[0, T ] can be deﬁned via eq. (2.3)

t1:= Yi

t2+Zt2

fi(s, Xs)ds −Zt2

(Zi

s)TdWs−Zt2

t1ZRd

∆Yi

s(e)˜

N(de, ds).

Approximation of certain continuation values will play a key role in approximating the value function of

interest. Our goal in the rest of this paper is to present an eﬃcient algorithm for calculating Yiwhere Xis a

high-dimensional state process with ﬁnite-variational jumps. In section 3, we ﬁrst provide some background

on neural networks, then we present the details of the Optimal Switching with Jumps (OSJ) algorithm.

3. Optimal Switching with Jumps (OSJ) Algorithm

3.1. Neural Network Structure. We utilize feedforward neural networks, which are in essence a series

of weighted sums of inputs composed with simple functions in such a way that unknown functions can be

approximated. Training data enters the network in the ﬁrst layer, and at each layer a weighted sum of the

inputs is computed using the choice of parameters assigned to the nodes in that layer to create an aﬃne

function. The output of each layer is processed by an activation function before becoming the input of the

next layer, and the ﬁnal layer produces the desired output of the network.

For a network of depth δwith δℓnodes in layer ℓ, there are Pδ−1

ℓ=0 δℓ(δℓ+1 + 1) = ¯

δparameters, represented

as a whole as θ. This θis chosen from all possible parameters in the parameter space Θδ, a compact subset

of R¯

δdeﬁned as

Θδ:= {θ∈R¯

δ,||θ||∞≤γδ},

where γδis positive and chosen to be very large. We can then deﬁne the set of neural networks that we are

working with as the union over δ∈Nof all the neural networks of depth δwith ¯

δtotal parameters. This

formulation accomplishes two things. First, the universal approximation theorem of [25] asserts that this set

of neural networks is dense in the set of continuous and measurable functions which map from Rd→Rs, for

any dimension s, and so are universally good approximators. Second, the parameter space associated with

this union, Θ = ∪δ∈NΘδ, represents the set of all possible weights that can be assigned to the nodes in the

neural network and is compact. Therefore, when trying to minimize the loss function associated with our

problem (which will be described in the next subsection), a minimizing θ∗exists.

The network therefore “learns” the function of interest by adjusting θvia multiple iterations of an opti-

mization algorithm. In our work, we use the Adam optimizer [27] applied to a four-layer neural network

with d+ 10 nodes in each layer and tanh as the chosen activation function. We ﬁx the input dimension as

d, and set the output dimension as d1= 1 + d+ 1 because Yi

t∈R, Zi

t∈Rd, and ∆Yi

t∈R.

3.2. Algorithm. To perform the numerical calculations, we discretize the continuous time interval [0, T ]

using a regular grid π={tn}M

n=0 ={nT/M}M

n=0, where T /M = ∆t. We denote the paths of the discrete

approximation as Xπand generate a large number of paths of Xπstarting from a desired initial condition

x0. We later use these paths as training data for the neural networks. At this point, we do not impose a

speciﬁc approximation scheme, but require that the in-time convergence be of at least strong order 0.5 for

convergence of the neural network value function in theorem 1 and of strong order 1.0 to achieve an auxiliary

A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS3

result regarding the performance of the neural network-generated strategies given in theorem 2. We describe

the approximation schemes used in our speciﬁc examples within the expository portions of section 4.1 and

section 4.2. In-depth discussion of weak- and strong-order approximations of jump-diﬀusion processes can

be found in [28].

We also discretize the switching times, introducing a grid Rwhere the grid spacing is of size T /√M,

meaning that |R| ∼ O(M−1/2). The process is able to switch modes at time tnwhere tn∈R, while evolving

as an uncontrolled jump-diﬀusion process when tn/∈R.

The continuation values between time steps of the grid πwill be learned by the neural network on a

mode-by-mode basis. We denote the neural network that learns the continuation value at tnin mode i

as Yi

n(Xπ

n, θi

1), the neural network that learns the derivative of the value function at the same stage as

n(Xπ

n, θi

2), and the neural network that learns the jump sizes for Yas ∆Yi

n(Xπ

n, θi

3). In practice, these

neural networks are treated as one larger network with combined parameter vector θi= (θi

1, θi

2, θi

3)∈Θ=Θδ

for some δcorresponding to the chosen architecture of the neural network. The functions generated by the

optimal choice of θiare deﬁned as

(3.1) 









n(Xπ

n) := Yi

n(Xπ

n, θ∗,i

n,1),

n(Xπ

n) := Zi

n(Xπ

n, θ∗,i

n,2),

∆Yi

n(Xπ

n) := ∆Yi

n(Xπ

n, θ∗,i

n,3).

The continuation values ˜

n(·) are then used to calculate the value functions

(3.2) ˆ

n(Xπ

n) := 1tn∈Rmax ˜

n(Xπ

n),max

j̸=i(˜

n(Xπ

n)−Ci,j (Xπ

n))+1tn/∈R˜

n(Xπ

The algorithm in its entirety is described in algorithm 1.

Algorithm 1 OSJ Algorithm

1: Generate paths of the stochastic process {Xπ

n}M

n=0 as well as ∆Wn:= Wtn+1 −Wtnand ∆ ˜

Nn:=

Rtn+1

tnRRd˜

N(de, ds) for each sample path. Store as training data.

2: Train ˆ

M≡gi(x),∀i∈I.

3: for n=M−1,...,2,1,0do

4: for all i∈Ido

5: Train a neural network to ﬁnd θ∗,i

n= (θ∗,i

n,1, θ∗,i

n,2, θ∗,i

n,3)∈Θ which minimizes

n(θ) = Eˆ

n+1(Xπ

n+1)− Yi

n(Xπ

n, θ1)

+fi(tn, Xπ

n)∆t− Zi

n(Xπ

n, θ2)∆Wn−∆Yi

n(Xπ

n, θ3)∆ ˜

Nn

(3.3)

6: Deﬁne ˜

n(·), ˆ

n(·), and d

∆Yi

n(·) as in eq. (3.1).

7: end for

8: for all i∈Ido

9: Calculate ˆ

n(·) as in eq. (3.2).

10: end for

11: The value function of interest is V(0, x0, i) = ˆ

0(x) where X0=xand α−1=i.

12: end for

The switching strategy arising from this algorithm is denoted aN N,M when the number of steps is chosen

to be M. This strategy is a function of the value of the state variable and starting mode, such that the

optimal mode at time tnis

αNN

n:= arg max

j∈I˜

n(·)−CαNN

n−1,j (·),

where the optimal mode at time tn−1is αNN

n−1∈Iand αNN

−1=α−1=i, the starting mode.

Remark 2. For theorem 2 we require that the discrete approximation of (Xt)t∈[0,T ]is of strong order 1.0

(instead of strong order 0.5 which is suﬃcient for theorem 1). This excludes the standard Euler–Maryama

discretization strategy, but there are a variety of other options. One good choice is a jump-adapted strong

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ANEURALNETWORKAPPROACHTOHIGH-DIMENSIONALOPTIMALSWITCHINGPROBLEMSWITHJUMPSINENERGYMARKETSERHANBAYRAKTAR,ASAFCOHEN,ANDAPRILNELLISAbstract.Wedevelopabackward-in-timemachinelearningalgorithmthatusesasequenceofneuralnetworkstosolveoptimalswitchingproblemsinenergyproduction,whereelectricityandfossilfuelpr...

展开>> 收起<<

A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS ERHAN BAYRAKTAR ASAF COHEN AND APRIL NELLIS.pdf

共27页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

A NEURAL NETWORK APPROACH TO HIGH-DIMENSIONAL OPTIMAL SWITCHING PROBLEMS WITH JUMPS IN ENERGY MARKETS ERHAN BAYRAKTAR ASAF COHEN AND APRIL NELLIS

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: