LQGNET HYBRID MODEL-BASED AND DATA-DRIVEN LINEAR QUADRATIC STOCHASTIC CONTROL Solomon Goldgraber Casspi Oliver H usser Guy Revach and Nir Shlezinger

2025-05-06 0 0 349.77KB 5 页 10玖币

侵权投诉

LQGNET: HYBRID MODEL-BASED AND DATA-DRIVEN LINEAR QUADRATIC

STOCHASTIC CONTROL

Solomon Goldgraber Casspi, Oliver H¨

usser, Guy Revach, and Nir Shlezinger

ABSTRACT

Stochastic control deals with ﬁnding an optimal control

signal for a dynamical system in a setting with uncertainty,

playing a key role in numerous applications. The linear

quadratic Gaussian (LQG) is a widely-used setting, where the

system dynamics is represented as a linear Gaussian state-

space (SS) model, and the objective function is quadratic.

For this setting, the optimal controller is obtained in closed

form by the separation principle. However, in practice, the

underlying system dynamics often cannot be faithfully cap-

tured by a fully known linear Gaussian SS model, limiting its

performance. Here, we present LQGNet, a stochastic con-

troller that leverages data to operate under partially known

dynamics. LQGNet augments the state tracking module of

separation-based control with a dedicated trainable algo-

rithm. The resulting system preserves the operation of classic

LQG control while learning to cope with partially known SS

models without having to fully identify the dynamics. We

empirically show that LQGNet outperforms classic stochastic

control by overcoming mismatched SS models.

Index Terms—Stochastic control, LQG, deep learning.

1. INTRODUCTION

Stochastic optimal control is a sub-ﬁeld of mathematical opti-

mization with applications spanning from operations research

to physical sciences and engineering, including aerospace, ve-

hicular systems, and robotics [1]. Stochastic control consdiers

dynamical system under the existence of uncertainty, either in

its evolution or in its observations.The aim is to ﬁnd an opti-

mal control signal for a given objective function. In the funda-

mental linear quadratic Gaussian (LQG) setting [2], e the sys-

tem dynamics obey a linear Gaussian state-space (SS) model,

and the controller should minimize a quadratic objective. The

optimal LQG controller follows the separation principle [3,4],

where state estimation is decoupled from control, and it com-

prises a Kalman ﬁlter (KF) followed by a conventional linear

quadratic regulator (LQR) [5].

While LQG control is simple and tractable, it relies on the

ability to faithfully describe the dynamics as a fully known

S. Goldgraber Casspi and N. Shlezinger are with the School of

ECE, Ben-Gurion University of the Negev, Be‘er Sheva, Israel (e-mail:

casspi@post.bgu.ac.il; nirshl@bgu.ac.il). O. H¨

usser and G. Revach are with

the D-ITET, ETH Zurich, Switzerland, (email: huesseol@student.ethz.ch;

grevach@ethz.ch). We thank Hans-Andrea Loeliger for helpful discussions.

linear Gaussian SS model. In practice, SS models are of-

ten approximations of the system’s true dynamics, while its

stochasticity can be non-Gaussian. The presence of such mis-

matched domain knowledge notably affects the performance

of classical model-based policies.

To overcome the drawbacks of oversimpliﬁed modeling,

one can resort to learning. The main learning-based approach

in sequential decision making and control is reinforcement

learning (RL) [6,7] where an agent is trained via experience-

driven autonomous learning to maximize a reward [8]. The

growing popularity of model-agnostic deep neural networks

(DNNs) and their empirical success in various tasks involv-

ing complex data, such as visual and language data, has led to

a growing interest in deep RL [9]. Deep RL systems based

on black-box DNNs were proposed for implementing con-

trollers for various tasks, including robotics and vehicular sys-

tems [10,11]. Despite their success, these architectures can-

not naturally incorporate the domain knowledge available in

partially known SS models, are complex and difﬁcult to train,

and lack the interpretability of model-based methods [12]. An

alternative approach uses DNNs to extract features processed

with model-based methods [13–15]. This approach still re-

quires one to impose a fully known SS model of the features,

motivating the incorporation of deep learning into classic con-

trollers to bypass the need to fully characterize the dynamics.

In this work, we propose LQGNet, a hybrid stochastic

controller designed via model-based deep learning [12,16].

LQGNet preserves the structure of the optimal LQG pol-

icy, while operating in partially known settings. We adopt

the recent KalmanNet architecture [17], which implements

a trainable KF, in a separation-based controller. The re-

sulting LQGNet architecture utilizes the available domain

knowledge by exploiting the system’s description as a SS

model, thus preserving the simplicity and interpretability of

the model-based policy while leveraging data to overcome

partial information and model mismatches. By converting

the optimal model-based LQG controller into a trainable dis-

criminative algorithm [18] our LQGNet learns to control

in an end-to-end manner. We empirically demonstrate that

LQGNet approaches the performance of the optimal LQG

policy with fully known SS models, and notably outperforms

it in the presence of mismatches.

The rest of this paper is organized as follows: Section 2

reviews the SS model and details the LQG control task. Sec-

tion 3presents LQGNet, which is evaluated in Section 4.

arXiv:2210.12803v2 [eess.SY] 25 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LQGNET:HYBRIDMODEL-BASEDANDDATA-DRIVENLINEARQUADRATICSTOCHASTICCONTROLSolomonGoldgraberCasspi,OliverH¨usser,GuyRevach,andNirShlezingerABSTRACTStochasticcontroldealswithndinganoptimalcontrolsignalforadynamicalsysteminasettingwithuncertainty,playingakeyroleinnumerousapplications.ThelinearquadraticGau...

展开>> 收起<<

LQGNET HYBRID MODEL-BASED AND DATA-DRIVEN LINEAR QUADRATIC STOCHASTIC CONTROL Solomon Goldgraber Casspi Oliver H usser Guy Revach and Nir Shlezinger.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

LQGNET HYBRID MODEL-BASED AND DATA-DRIVEN LINEAR QUADRATIC STOCHASTIC CONTROL Solomon Goldgraber Casspi Oliver H usser Guy Revach and Nir Shlezinger

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: