Actor-Critic Network for O-RAN Resource Allocation xApp Design Deployment and Analysis Mohammadreza Kouchaki and Vuk Marojevic

2025-04-27 0 0 2.05MB 6 页 10玖币

侵权投诉

Actor-Critic Network for O-RAN Resource

Allocation: xApp Design, Deployment, and Analysis

Mohammadreza Kouchaki and Vuk Marojevic

Electrical and Computer Engineering, Mississippi State University, USA

mk1682@msstate.edu, vm602@msstate.edu

Abstract—Open Radio Access Network (O-RAN) has intro-

duced an emerging RAN architecture that enables openness,

intelligence, and automated control. The RAN Intelligent Con-

troller (RIC) provides the platform to design and deploy RAN

controllers. xApps are the applications which will take this

responsibility by leveraging machine learning (ML) algorithms

and acting in near-real time. Despite the opportunities provided

by this new architecture, the progress of practical artiﬁcial intel-

ligence (AI)-based solutions for network control and automation

has been slow. This is mostly because of the lack of an end-

to-end solution for designing, deploying, and testing AI-based

xApps fully executable in real O-RAN network. In this paper we

introduce an end-to-end O-RAN design and evaluation procedure

and provide a detailed discussion of developing a Reinforcement

Learning (RL) based xApp by using two different RL approaches

and considering the latest released O-RAN architecture and

interfaces.

Index Terms—O-RAN, Near Real-Time RIC, xApp, Resource

Allocation, Reinforcement Learning, Actor-Critic, AI.

I. INTRODUCTION

Telecommunication networks will soon provide wireless

connectivity and facilitate tiny and huge data transactions

among 10s of billions of smart devices. Resource, data,

and network management is becoming more challenging and

artiﬁcial intelligence (AI) solutions are being researched for

facilitating future wireless network operations. The major ob-

stacles are the resource restrictions and the lack of a proﬁcient

platform to handle AI solutions completely independently

from the network hardware, decreasing the cost of changes to

new third-party software solutions [1]. The hardware, software,

and interfaces of traditional radio access networks (RANs) are

tightly coupled. Recent advancements of RAN technology can

help breaking such closed designs and vendor monopoly [2].

A new architecture introduced by the Open-RAN (O-RAN)

Alliance can bring this idea into reality and change the future

of RAN deployment, operation, and maintenance [3].

The O-RAN Alliance deﬁnes speciﬁcations to facilitate AI

integration and allow machines and software to function intel-

ligently in a cellular network. The O-RAN architecture enables

intelligence and openness by providing an infrastructure for

integrating RANs on open hardware with embedded AI-based

software [4]. This architecture supports the Third Generation

Partnership Project (3GPP) and other industry standards. 3GPP

deﬁnes the radio and network protocols for the user equipment,

RAN, and core network. O-RAN leverages those and the Radio

Unit (RU), Distributed Unit (DU), and Centralized Unit (CU)

that 3GPP deﬁnes and speciﬁes particular RAN functional

splits and open interfaces facilitating practical disaggregation

of functionalities and integration from different vendors. O-

RAN also introduces new architectural components: the Near-

Real Time (RT) RAN Intelligent Controller (RIC), the Non-RT

RIC, and additional interfaces which pave the way for insert-

ing intelligent network control and optimization applications

called xApps [5].

The challenges for developing xApps and deploying them

on real networks include: ﬁnding the most efﬁcient AI models

suitable for very large real-world networks, adopting the most

proﬁcient network parameters, and testing the AI models in an

environment that accurately represents the behavior of real-

world networks. I light of these concerns, this paper details

the development and testing ﬂow of a reinforcement learning

(RL) based xApp based on O-RAN architecture. We discuss

a O-RAN architectural components, interfaces, and workﬂow

to design an xApp. We investigate and simulate different AI

solutions to analyze the performance of various RL methods

for designing the xApp.

The rest of this paper is organized as follow: Section II

introduces the O-RAN architecture and the main components

that are relevant to the AI controller design. Section III

presents the related work. Section IV discusses the design

procedure and the challenges for developing xApps. Section

V analyzes different AI models to select the most efﬁcient

solution. Sections VI describes the xApp development and

Section VII the deployment and results. Section VIII draws

the conclusions.

II. O-RAN ARCHITECTURE AND KEY COMPONENTS FOR

XAPP DEVELOPMENT

The network architecture needs to provide a platform for

deploying AI/ML-based applications and provide the required

infrastructure for data transactions from the RAN nodes to the

AI model, data storage, transmission of the model decision and

control commands to the network, and the AI model training

process. The O-RAN architecture shown in Fig. 1 is based on

open interfaces to enable interactions between the RAN and

the RAN controller. The RAN is split into three logical units:

CU, DU, and RU. The CU is a centralized unit developed

to handle the higher layer RAN protocols, such as the radio

resource control (RRC), the service data adaptation protocol

(SDAP), and the packet data convergence protocol (PDCP).

It interfaces with the DUs through the mid-haul. The DU is

a logical node that handles the lower protocol layers, which

arXiv:2210.04604v1 [cs.NI] 26 Sep 2022

Fig. 1. O-RAN architecture for RIC application layer.

are the radio link control (RLC), the medium access control

(MAC), and part of the physical layer (PHY). It interfaces

with the RUs through the fronthaul. The RU implements the

lower part of the PHY.

Data is transmitted from the RAN to the non-RT RIC

through the O1 interface and is stored in a database for

ofﬂine training and testing of the AI/ML model. The model

training will take place at the non-RT RIC which is also

responsible for performing non-RT control operations in O-

RAN and for providing and managing higher layer policies.

After training, the xApp will run on the near-RT RIC and

interact with the RAN through the E2 interface to perform

online optimization and control of the network. An xApp can

communicate with other parts of the near-RT RIC through

internal interfaces which are introduced in Section IV. There

is an internal messaging infrastructure called RIC Message

Router (RMR) and a shared data layer (SDL) for data sharing.

The near-RT RIC provides the framework to handle conﬂicts,

subscriptions, applications, security, and logging.

III. RELATED WORK

Several ML based schedulers have been introduced in the

literature to address the most challenging problem of resource

allocation in cellular networks. Gosal et al. introduce a cen-

tralized RL-based scheduler based on the Deep Deterministic

Policy Gradient (DDPG) considering pricing rate of resources

[6]. Elsayed et al. discuss challenges of AI-enabled solutions

for optimizing network resource orchestration [7]. Polese et

al. propose an ML-based edge-controller architecture for the

5G network and use generated data in a testbed to evaluate the

model [8]. Niknam et al. propose an ML based resource allo-

cation scheme for controlling O-RAN network congestion and

evaluate the model using published real-world data from an op-

erator [9]. Mollahasani et al. introduce an optimization method

for RL based solutions of resource allocation function in O-

RAN [10]. They investigate the effects of observation nodes on

the performance of the resource allocation. Mollahasani, at al.

design a RL based scheduler for allocating resource blocks in

a reconﬁgurable wireless network considering mobility [11].

Bonati et al. introduce an open experimental toolbox which

provides an open testbed for AI/ML xApps and present an

ML based scheduler and test results [12].

The prior works focus on the ML application, testing, and

evaluation parameters. This paper presents the detailed process

for designing and deploying AI/ML based xApps on the

RIC. The O-RAN Software Community (OSC) has published

several xApps developed by its members [13].

IV. XAPP DEVELOPMENT

To write any AI/ML solution in the format of an xApp to

be deployed in the RIC, two main steps should be considered.

The ﬁrst is to write an xApp with the essential libraries and

functions based on the RIC requirements. For this purpose,

developers can use the RIC utility libraries, such as the RMR,

SDL, logging, or use a predeﬁned xApp frameworks that have

been developed based on the RIC platform requirements. The

xApp frameworks simpliﬁes developing xApps in Python, go,

or C++ [13]. We have used the ricxappframe 3.2.0 provided

by PyPi for our development to facilitate adding essential

features such as communication functions with RMR and SDL.

The second step is building and deploying the application

on Kubernetes since the RIC cluster is developed on the

Kubernetes platform.

Kubernetes is an open-source platform for deploying and

managing containerized applications across clusters of nodes.

Fig. 2 shows the high-level Kubernetes architecture which

consists of a control plane, a master node, and a number

of worker nodes that execute the deployed applications. The

master node hosts the API server, scheduler for assigning

worker nodes to applications, etcd as a key-value distributed

storage system, and a controller manager.

The worker nodes which are running containerized applica-

tions are built of different components. The deployment unit

of Kubernetes is Pod, which is a group of containers with

shared resources. Pod with all of its containers is deployable

through a Yaml ﬁle that determine the Pod conﬁgurations such

as ports, name, the number of replicas and is implemented on

one machine that has a single IP address shared among all of

its containers. The next component is the container runtime

(e.g., Docker) that is responsible for running containers. The

Kubeproxy unit routes trafﬁc coming into a node from the ser-

vice and forwards work requests to the correct containers. To

provide communication with the master node and containers

of the worker node, kubernetes uses kubelet service that also

traces the states of a pod to check whether all the containers

are running.

After developing the main application code in Python within

the xApp framework, we need to deploy it as an containerized

application on Kubernetes. In RIC To deploy our containerized

application we use the ricxapp Pod in the Kubernetes node.

For xApp deployment we have four main steps: First we

should containerize the application to create a container image.

This facilitates porting an application and executing it on any

machine. To create a container image, we wrote a Docker ﬁle

that includes the instructions to run our Python code and built

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Actor-CriticNetworkforO-RANResourceAllocation:xAppDesign,Deployment,andAnalysisMohammadrezaKouchakiandVukMarojevicElectricalandComputerEngineering,MississippiStateUniversity,USAmk1682@msstate.edu,vm602@msstate.eduAbstractOpenRadioAccessNetwork(O-RAN)hasintro-ducedanemergingRANarchitecturethatenable...

展开>> 收起<<

Actor-Critic Network for O-RAN Resource Allocation xApp Design Deployment and Analysis Mohammadreza Kouchaki and Vuk Marojevic.pdf

共6页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Actor-Critic Network for O-RAN Resource Allocation xApp Design Deployment and Analysis Mohammadreza Kouchaki and Vuk Marojevic

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: