Fine-Grained Session Recommendations in E-commerce using Deep Reinforcement Learning

2025-05-06 0 0 622.43KB 6 页 10玖币

侵权投诉

Fine-Grained Session Recommendations in E-commerce using

Deep Reinforcement Learning

Diddigi Raghu Ram Bharadwaj

Myntra Designs Pvt. Ltd.

India

diddigi.bharadwaj@myntra.com

Lakshya Kumar

Myntra Designs Pvt. Ltd.

India

lakshya.kumar@myntra.com

Saif Jawaid

Myntra Designs Pvt. Ltd.

India

saif.jawaid@myntra.com

Sreekanth Vempati

Myntra Designs Pvt. Ltd.

India

sreekanth.vempati@myntra.com

ABSTRACT

Sustaining users’ interest and keeping them engaged in the plat-

form is very important for the success of an e-commerce business.

A session encompasses dierent activities of a user between logging

into the platform and logging out or making a purchase. User activ-

ities in a session can be classied into two groups: Known Intent

and Unknown intent. Known intent activity pertains to the session

where the intent of a user to browse/purchase a specic product can

be easily captured. Whereas in unknown intent activity, the intent

of the user is not known. For example, consider the scenario where

a user enters the session to casually browse the products over the

platform, similar to the window shopping experience in the oine

setting. While recommending similar products is essential in the

former, accurately understanding the intent and recommending in-

teresting products is essential in the latter setting in order to retain

a user. In this work, we focus primarily on the unknown intent set-

ting where our objective is to recommend a sequence of products to

a user in a session to sustain their interest, keep them engaged and

possibly drive them towards purchase. We formulate this problem

in the framework of the Markov Decision Process (MDP), a popular

mathematical framework for sequential decision making and solve

it using Deep Reinforcement Learning (DRL) techniques. However,

training the next product recommendation is dicult in the RL

paradigm due to large variance in browse/purchase behavior of the

users. Therefore, we break the problem down into predicting vari-

ous product attributes, where a pattern/trend can be identied and

exploited to build accurate models. We show that the DRL agent

provides better performance compared to a greedy strategy.

CCS CONCEPTS

•Computing methodologies →Sequential decision making

;

Reinforcement learning.

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specic permission and/or a

fee. Request permissions from permissions@acm.org.

KDD Workshop, Aug 15, 2022, Washington DC, USA

ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00

https://doi.org/XXXXXXX.XXXXXXX

KEYWORDS

Sequential Recommendation, Reinforcement Learning, Session In-

tent Prediction

ACM Reference Format:

Diddigi Raghu Ram Bharadwaj, Lakshya Kumar, Saif Jawaid, and Sreekanth

Vempati. 2022. Fine-Grained Session Recommendations in E-commerce

using Deep Reinforcement Learning. In Proceedings of 1st Workshop on End-

End Customer Journey Optimization (KDD Workshop). ACM, New York, NY,

USA, 6 pages. https://doi.org/XXXXXXX.XXXXXXX

1 INTRODUCTION

An e-commerce platform provides a promising alternative to the

traditional retail business where users can browse, compare, make

purchases and get delivery from the comfort of their homes. This

setting has become even more popular recently due to the pandemic,

where contactless purchase and delivery was preferred over the

traditional shopping experience. This increased digital footprint of

users on the platform led to an exponential growth of customer data

like browsing history, purchase history, session activity, etc. From

the e-commerce business point of view, it is paramount to make op-

timal use of this data not only to increase the prots of the business

but also to improve the customer experience by recommending

appropriate items.

Session recommendation is an essential subproblem in the rec-

ommendation system where the objective is to recommend the

items to the users primarily based on the user’s activity thus far in

the session. It diers from the traditional recommendation systems

that use the user’s historical browse/purchase interactions data.

On the contrary, each session is treated independently in session

recommendation problems. This is due to the diculty in learn-

ing the dependencies between the dierent sessions of the user.

Moreover, the sessions might not be correlated, and the user’s prior

history (before the session starts) is not always available for cold

start cases. Although appealing due to its simplicity, this paradigm

is challenging to solve due to the inherent heterogeneity in the

users’ activities across the platform and varying goals associated

with it. To understand this, consider the following example: In the

o-line shopping scenario, a sequence of items is suggested to the

customers by the sales executive based on the explicit feedback

provided by the customer. It is imperative that the executive sug-

gests the items to sustain the customers’ interest and ultimately

drive them towards a purchase. Each item that is suggested has an

arXiv:2210.15451v1 [cs.IR] 20 Oct 2022

KDD Workshop, Aug 15, 2022, Washington DC, USA Diddigi et al.

Figure 1: Proposed architecture of session recommendation.

The previous 𝑘(three in this example) products are consid-

ered for recommending product at instant 𝑘+

. First, the at-

tributes of the products (𝐴, 𝐵, 𝐶) are extracted. Next, these at-

tributes are then fed to the RL agents to generate the next at-

tribute recommendations. Finally, the attribute recommen-

dations are combined to obtain the product to be recom-

mended at instant 𝑘+1(fourth in this example).

immediate impact (i.e., customers might either like and continue

exploring it or exit the store) and also inuences the future actions

of the customers (i.e., making a purchase). Emulating this in the

online scenario requires capturing this dynamic nature of the prob-

lem and balancing short-term and long-term goals. This motivates

us to formulate it in the framework of the Markov Decision Process

(MDP) [

]. We dene a session as a sequence of events of a user

until it leads to one of the following (a). purchase (b). user exits

the session. This work aims to optimally recommend a sequence of

products that could potentially lead to a purchase event.

Reinforcement Learning (RL) [

] is a popular model-free para-

digm for solving an MDP problem. Here, we train an agent to make

optimal decisions based only on the trajectories of the environ-

ment. When the number of states and actions in the environment

is very high, one resorts to function approximation architectures.

RL algorithms combined with neural network architectures, i.e.,

Deep RL, have achieved a lot of success in recent times [

]. In

this work, we train a Deep RL agent to recommend a sequence of

products in a session. It is important to note that the traditional

RL setup where agents learn by exploring dierent actions is not a

favorable setting for our problem due to a large number of prod-

ucts in the e-commerce space. Hence, we train the algorithm under

an o-policy setting using the users’ historical session data. The

dataset considered in this work is compiled from the click-stream

data of users on the Myntra e-commerce platform, one of the largest

fashion e-commerce in India.

Training a deep RL agent to recommend the products directly is

not practical due to heterogeneity (in terms of dierent attributes)

in the users’ browsing history. There are two problems associated

with this training paradigm. First, the number of products is huge

(which constitutes the action space). Second, there might not be

a denite trend that can be learned from this data, making the

training very unstable. For example, consider a scenario where two

users browse products in a sequence that is similar in every regard

(like product type, color) except a specic attribute ‘brand.’ Say

the session of the former user ends up in a purchase, whereas the

latter is a non-purchase session. The eectiveness of RL training

lies in the generalization of actions and hence the sessions of such

nature will lead to unstable learning. To mitigate this problem, we

propose a divide-and-conquer approach where multiple RL agents

will be trained to recommend various attributes of the products, and

these recommendations will be combined at the end to generate

product recommendations. This is illustrated in Figure 1. First,

attributes of the products (like color, product type, and brand) are

extracted. These attributes are then sent as inputs to independent

DRL models to obtain the recommendations for attributes in the

next time instant. Finally, the products that match the attributes

are presented as nal recommendations to the user.

The overall contributions of the paper are as follows:

•

We mathematically formulate the problem of User Session

recommendation in the framework of MDP.

•

We propose a Deep Q-Learning based model to predict the

next products within a user session while optimizing for

purchase intent.

•

We compare the proposed model with a similarity-based

baseline model to showcase our proposed approach’s e-

cacy.

2 RELATED WORK

Recommendation systems deal with building algorithms for rec-

ommending products to the user to meet various objectives like

user personalization, increased engagement rate, and improving

business goals. The idea here is to accurately predict users’ interest

and recommend products that meet their expectations. Recom-

mendation systems nds its applications in various domains like

news recommendation [

], movie recommendation

[

] etc. The importance of recommendation systems is

even more pronounced in the e-commerce business, where the buy-

ing and selling of products are performed virtually online. There-

fore, it is imperative from the business point of view to recommend

relevant and specic products to the users to sustain their interest

over a long period. As a result, a lot of research has been dedicated

to build good recommendation systems [

] in recent times to solve

problems like Click-Through-Rate (CTR) prediction [

intent and purchase prediction [6, 13, 37] etc.

Deep Learning is a popular class of machine learning algorithms

that uses articial neural networks to learn and derive required

patterns from the input data. We will now discuss some popular

deep learning techniques proposed in the literature for solving the

recommendation problem. In [

], two deep learning algorithms

based on the collaborative ltering technique have been proposed

to handle cold-start problems. In [

], a deep learning model has

been deployed to simulate the interaction between item and user by

feeding the pre-trained representations of item and user as input to

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Fine-GrainedSessionRecommendationsinE-commerceusingDeepReinforcementLearningDiddigiRaghuRamBharadwajMyntraDesignsPvt.Ltd.Indiadiddigi.bharadwaj@myntra.comLakshyaKumarMyntraDesignsPvt.Ltd.Indialakshya.kumar@myntra.comSaifJawaidMyntraDesignsPvt.Ltd.Indiasaif.jawaid@myntra.comSreekanthVempatiMyntraDesi...

展开>> 收起<<

Fine-Grained Session Recommendations in E-commerce using Deep Reinforcement Learning.pdf

共6页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Fine-Grained Session Recommendations in E-commerce using Deep Reinforcement Learning

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: