QUANTUM DEEP RECURRENT REINFORCEMENT LEARNING Samuel Yen-Chi Chen Wells Fargo

2025-04-29 0 0 1.16MB 5 页 10玖币

侵权投诉

QUANTUM DEEP RECURRENT REINFORCEMENT LEARNING

Samuel Yen-Chi Chen

Wells Fargo

ABSTRACT

Recent advances in quantum computing (QC) and machine

learning (ML) have drawn signiﬁcant attention to the devel-

opment of quantum machine learning (QML). Reinforcement

learning (RL) is one of the ML paradigms which can be used to

solve complex sequential decision making problems. Classical

RL has been shown to be capable to solve various challenging

tasks. However, RL algorithms in the quantum world are still

in their infancy. One of the challenges yet to solve is how to

train quantum RL in the partially observable environments.

In this paper, we approach this challenge through building

QRL agents with quantum recurrent neural networks (QRNN).

Speciﬁcally, we choose the quantum long short-term memory

(QLSTM) to be the core of the QRL agent and train the whole

model with deep

-learning. We demonstrate the results via

numerical simulations that the QLSTM-DRQN can solve stan-

dard benchmark such as Cart-Pole with more stable and higher

average scores than classical DRQN with similar architecture

and number of model parameters.

Index Terms—

Quantum machine learning, Reinforce-

ment learning, Recurrent neural networks, Long short-term

memory

1. INTRODUCTION

Quantum computing (QC) promises superior performance on

certain hard computational tasks over classical computers [

However, existing quantum computers are not error-corrected,

making implementation of deep quantum circuits extremely

difﬁcult. These so-called noisy intermediate-scale quantum

(NISQ) devices [

] require special design of quantum circuit

architectures so that the quantum advantages can be harnessed.

Recently, a hybrid quantum-classical computing framework

[

] which leverage both the classical and quantum computing

has been proposed. Under this paradigm, certain computa-

tional tasks which are expected to have quantum advantages

are carried out on a quantum computer, while other tasks

such as gradient calculations remain on the classical comput-

ers. These algorithms are usually called variational quantum

The views expressed in this article are those of the authors and do not

represent the views of Wells Fargo. This article is for informational purposes

only. Nothing contained in this article should be construed as investment

advice. Wells Fargo makes no express or implied warranties and expressly

disclaims all legal, tax, and accounting implications related to this article.

algorithms are have been successful in certain ML tasks. Re-

inforcement learning (RL) is a sub-ﬁeld of ML dealing with

sequential decision making tasks. RL based on deep neural

networks have gained tremendous success in complex tasks

with human-level [

] or super-human performance [

]. How-

ever, quantum RL is an emerging subject with many issues

and challenges not yet investigated. For example, existing

quantum RL methods focus on various VQCs without the re-

current structures. Nevertheless, recurrent connections are

crucial components in the classical ML to keep memory of

past time steps. The potential of such architectures, to our

best knowledge, is not yet studied in the quantum RL. In this

work, we propose the quantum deep recurrent

-learning via

the application of quantum recurrent neural networks (QRNN)

as the value function approximator. Speciﬁcally, we apply the

quantum long short-term memory (QLSTM) as the core of the

QRL agent. The scheme is illustrated in Figure1. Our numeri-

cal simulation shows that the proposed framework can reach

performance comparable to or better than their classical LSTM

counterparts when the model sizes are similar and under the

same training setting.

Quantum Recurrent RL

Environment

Hybrid RL Agent

action

state

reward

Quantum RNN Classical Computer

Deep Q-learning

algorithm

Update

parameters

Output from

Quantum

RNN

Fig. 1

The hybrid quantum-classical deep recurrent Q-

learning.

2. RELATED WORK

The quantum reinforcement learning (QRL) can be traced back

to the work [

]. However, the framework requires the envi-

ronment to be quantum, which may not be satisﬁed in most

real-world cases. Here we focus on the recent developments

of VQC-based QRL dealing with classical environments. The

ﬁrst VQC-based QRL [

], which is the quantum version of

arXiv:2210.14876v1 [quant-ph] 26 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

QUANTUMDEEPRECURRENTREINFORCEMENTLEARNINGSamuelYen-ChiChenWellsFargoABSTRACTRecentadvancesinquantumcomputing(QC)andmachinelearning(ML)havedrawnsignicantattentiontothedevel-opmentofquantummachinelearning(QML).Reinforcementlearning(RL)isoneoftheMLparadigmswhichcanbeusedtosolvecomplexsequentialdecisio...

展开>> 收起<<

QUANTUM DEEP RECURRENT REINFORCEMENT LEARNING Samuel Yen-Chi Chen Wells Fargo.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

QUANTUM DEEP RECURRENT REINFORCEMENT LEARNING Samuel Yen-Chi Chen Wells Fargo

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: