Maximum principle for discrete time mean-eld stochastic optimal control problems Arzu Ahmadova1 Nazim I. Mahmudov2

2025-05-02 0 0 382.07KB 25 页 10玖币

侵权投诉

Maximum principle for discrete time mean-ﬁeld

stochastic optimal control problems

Arzu Ahmadova1, Nazim I. Mahmudov2

1Faculty of Mathematics, University of Duisburg-Essen, 45127, Essen, Germany,

e-mail: arzu.ahmadova@uni −due.de

2Department of Mathematics, Eastern Mediterranean University, 99628, T.R. North

Cyprus,

e-mail: nazim.mahmudov@emu.edu.tr

Abstract

In this paper, we study the optimal control of a discrete-time stochastic dif-

ferential equation (SDE) of mean-ﬁeld type, where the coeﬃcients can depend

on both a function of the law and the state of the process. We establish a new

version of the maximum principle for discrete-time stochastic optimal control

problems. Moreover, the cost functional is also of the mean-ﬁeld type. This

maximum principle diﬀers from the classical principle since we introduce new

discrete-time backward (matrix) stochastic equations. Based on the discrete-

time backward stochastic equations where the adjoint equations turn out to

be discrete backward SDEs with mean ﬁeld, we obtain necessary ﬁrst-order

and suﬃcient optimality conditions for the stochastic discrete optimal control

problem. To verify, we apply the result to production and consumption choice

optimization problem.

Keywords: Discrete time stochastic maximum principle, backward stochas-

tic diﬀerence equations, mean-ﬁeld theory, optimal control problem, necessary

and suﬃcient conditions

Contents

1 Introduction 2

arXiv:2210.01197v1 [math.OC] 3 Oct 2022

2 Mathematical description 5

3 Statement of main results 6

4 Backward Stochastic Diﬀerence Equations 9

5 Proof of Theorem 2 10

5.1 Dualityanalysis.............................. 12

6 Suﬃcient conditions for optimality 17

7 Application 19

8 Conclusions 21

1 Introduction

A large number of problems, interesting from a theoretical point of view and im-

portant from a practical one, has attracted the attention of many mathematicians

and engineers. It is not surprising that there is no ﬁeld in which extremal problems

do not arise, and in which it is not essential to the development of these ﬁelds that

such problems should be solved. The development of the necessary conditions for

an extremum was the elaboration of convex programming theory. A central place in

this theory is occupied by the Kuhn-Tucker theorem. The embedding of the theory

of optimal control in a general theory of necessary conditions was ﬁrst carried out

by Milyutin and Dubovitskii [1]. The great importance of their work lies in the

fact that they succeeded in formulating in a reﬁned form necessary conditions for an

extremum which can be applied to a wide class of problems.

The maximum principle for discrete-time systems has become a subject of great

interest. We begin this section by summarizing some seminal articles in this ﬁeld.

In [3] it was shown that the convexity requirement is not applicable to many practical

systems. Holtzman and Halkin [4] extend the applicability to much broader classes

of practical systems under the condition of directional convexity being weaker than

convexity. Moreover, Gamkrelidze [12] proved a maximum principle for systems with

phase constraints under a number of assumptions. A number of original ideas related

to proving the maximum principle can be found in the works of Rozenoer [6].

Jordon and Polak [7] have also considered the problem for optimal discrete sys-

tems and derived a stationary principle. They applied similar arguments to those

used in deriving the Pontryagin maximum principle for continuous-time systems [5].

Butkovski [8] ﬁrst showed that, in contrast to the continuous case, a direct extension

of Pontryagin’s maximum principle to discrete systems is in general impossible. Of

course, such a property of these systems is of theoretical interest to researchers. He

clearly demonstrated some errors in the existing works. The intrinsic reason for the

errors is that the signiﬁcance of convexity has been ignored. Generally speaking,

the discrete-time maximum principle fails unless a certain convexity precondition is

imposed on the control system. However, in this connection, some researchers have

established additional conditions, such as convexity of the set of admissible velocities

of the system, directional convexity and z-directional convexity, etc., and found that

under these conditions the maximum principle is valid for discrete control systems.

Many results have been done on this topic for diﬀerent kinds of continuous-

time stochastic optimal control problems, for example [9, 10, 14, 17, 25, 27–29], and

discrete-time stochastic optimal control problems, see [?,11,13,16,18–22,26] and the

references therein). The main diﬃculty of the stochastic maximum principle for an

optimal control problem governed by continuous-time stochastic Itˆo equations is that

the stochastic Itˆo integral is only of order ε(”hidden convexity” fails). Therefore,

the usual method of ﬁrst-order needle variation fails. To overcome this diﬃculty,

one has to study both the ﬁrst and second order terms in the Taylor expansion

of the needle variation, and establish a stochastic maximum principle consisting of

two backward stochastic diﬀerential equations and a maximum condition with an

additional quadratic term in the diﬀusion coeﬃcients, see [14,15]. It should be noted

that Lin and Zhang [21] used spike variations to show that the necessary condition

for discrete-time stochastic optimal problems is associated with the solutions of a

pair of discrete-time backward stochastic equations. On this basis, they obtained

the maximum principle for the discrete-time stochastic optimal control problem.

As for the discrete maximum principle for mean-ﬁeld stochastic optimal control

problems framework, there are a few papers dealing with discrete-time mean-ﬁeld

stochastic optimal control. Unlike the classical stochastic control problem, mean-

ﬁeld terms appear in the system dynamics and cost function, connecting mean-ﬁeld

theory to stochastic control problems. The stochastic mean-ﬁeld control problem has

been an important research topic since the 1950s. The system state is described by

a controlled mean-ﬁeld stochastic diﬀerential equation (MF-SDE), which was ﬁrst

proposed in [5], and the ﬁrst study on MF-SDEs was published in [2]. Since then,

many researchers have made numerous contributions to the study of MF-SDEs and

related topics, see, e.g. [3, 9, 11,15, 23,24] and the references cited therein.

Among the many scientiﬁc articles on discrete stochastic maximum principle, we

will mention only a few with comparison and relation that motivate this work:

•Song and Liu considered in [10] the optimal control problem for fully cou-

pled forward–backward stochastic diﬀerence equations of mean-ﬁeld type under

weak convexity assumption. Note that the form of (3.6) as an adjoint equa-

tion which was introduced in [10] is one kind of backward stochastic diﬀerence

equation. This adjoint equation is quite diﬀerent from our adjoint equation

(3) studied in this paper. One the one hand, they have diﬀerent forms, on the

other hand, the adjoint equation (3.6) is Ft+1-measurable;

•In [26], Wu and Zhang studied recently discrete-time stochastic optimal control

problem with convex control domains, for which necessary condition in the form

of Pontryagin’s maximum principle and suﬃcient condition of optimality are

derived. They also pointed out that how to overcome of integrability problem

of the solution to the adjoint equation which was not taken into consideration

in [10].

•Recently, Mahmudov [13] derived the ﬁrst-order and second-order necessary

optimality conditions for discrete-time stochastic optimal control problems

by virtue of new discrete-time backward stochastic equation and backward

stochastic matrix equation under assumption of the set

(f, σ1, σ2, . . . , σd, l)(t, ¯x(t), U(t)) being convex. Unlike [13] and [26] we study

mean-ﬁeld type discrete-time stochastic maximum principle.

Based on the above considerations, the main purpose of this paper is to construct

a rigorous mathematical framework for a mean-ﬁeld type of discrete-time stochastic

optimal control problems and to obtain a rigorous maximum principle in an under-

standable way. We study the maximum principle for the optimal control of discrete-

time systems described by mean-ﬁeld stochastic diﬀerence equations. As far as we

know, there are few results on such stochastic control problems. In fact, discrete-time

control systems are of great value in practice. For example, digital control can be for-

mulated as a discrete-time control problem in which the sampled data are obtained

at discrete times. In a discrete-time system, the Riccati diﬀerence equation plays

an important role in synthesizing the optimal control. As pointed out in [26], the

integrability of the solution to the adjoint equation in discrete-time stochastic opti-

mal control problem is completely diﬀerent from that in the continuous-time case.

However, we also prove that the solution of the adjoint equation has no problem of

integrability.

The main perspectives of our work are systematized as below:

•First, to study discrete stochastic optimal control problems, we use the ﬁ-

nite approximation method applied in [20]. We extend this method to study

the discrete-time stochastic backward equation and introduce the discrete-time

stochastic backward matrix equation;

•Second, we prove that the solution to the adjoint equation has no problem of

integrability;

•Next, a constructive method is that when the necessary optimality condition

are also suﬃcient under certain assumptions;

•Finally, as an application, we adapt the practical application based on Theo-

rem 2 and consider the discrete-time system with some risk in the investment

process.

The structure of the paper is as follows. In Section 2, we formulate the main results

and give an example to show the applicability of our results. Section 3 is devoted to

stating main results of this paper. In Section 4, we introduce the discrete-time back-

ward stochastic equation and the discrete-time backward stochastic matrix equation

and present the solutions in terms of a fundamental stochastic matrix. In Section

5, we prove the discrete-time stochastic maximum principle: a ﬁrst-order necessary

condition for optimality. Section 6 is devoted to the suﬃcient condition for optimal-

ity. Section 7 is devoted to the application of production and consumption choice

optimization problems.

2 Mathematical description

In Section 2 we present in Setting 1 the mathematical framework which we use to

study the discrete-time stochastic optimal control problems of mean-ﬁeld type.

Setting 1. Let k·k be a norm, h·,·i be an inner product, let n1, n2∈Nand denote

the space of (n1×n2)-matrices by Rn1×n2, and let Rn1:=Rn1×1, that is, each element

of Rn1is understood as a column vector, let Ibe the unit matrix with appropriate

dimension. For each matrix A,A|denotes the transpose of A. Moreover, the for-

ward diﬀerence operator ∆is deﬁned for all h > 0as ∆f(t) = f(t+h)−f(t). For

a vector x∈Rndenote by x|its transpose. For a symmetric matrix Aand vec-

tors y, y1, y2of matching dimensions, we denote A[y]2:=y|Ay,A[y1, y2]:=y|

1Ay2,

f[t]:=f(t, bx(t),Ebx(t),bu(t)).

Let (Ω,F,P)be a complete probability space and Nbe a positive integer. T:=

{tk=t0+kh, h > 0}N

k=0, let {w(tk) : k= 1, . . . , N + 1}be a sequence of Fk-measurable

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Maximumprinciplefordiscretetimemean-eldstochasticoptimalcontrolproblemsArzuAhmadova1,NazimI.Mahmudov21FacultyofMathematics,UniversityofDuisburg-Essen,45127,Essen,Germany,e-mail:arzu:ahmadova@unidue:de2DepartmentofMathematics,EasternMediterraneanUniversity,99628,T.R.NorthCyprus,e-mail:nazim:mahmudov...

展开>> 收起<<

Maximum principle for discrete time mean-eld stochastic optimal control problems Arzu Ahmadova1 Nazim I. Mahmudov2.pdf

共25页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Maximum principle for discrete time mean-eld stochastic optimal control problems Arzu Ahmadova1 Nazim I. Mahmudov2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: