Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders

2025-05-02 0 0 1.06MB 10 页 10玖币

侵权投诉

Learning Vector-antized Item Representation for

Transferable Sequential Recommenders

Yupeng Hou

houyupeng@ruc.edu.cn

Gaoling School of Articial Intelligence

Renmin University of China

Beijing, China

Zhankui He

zhh004@eng.ucsd.edu

UC San Diego

San Diego, California, USA

Julian McAuley

jmcauley@eng.ucsd.edu

UC San Diego

San Diego, California, USA

Wayne Xin Zhao†B

batmany@gmail.com

Gaoling School of Articial Intelligence

Renmin University of China

Beijing, China

ABSTRACT

Recently, the generality of natural language text has been leveraged

to develop transferable recommender systems. The basic idea is

to employ pre-trained language models (PLM) to encode item text

into item representations. Despite the promising transferability,

the binding between item text and item representations might be

too tight, leading to potential problems such as over-emphasizing

the eect of text features and exaggerating the negative impact of

domain gap. To address this issue, this paper proposes

VQ-Rec

, a

novel approach to learning

ector-

uantized item representations

for transferable sequential

Rec

ommenders. The main novelty of

our approach lies in the new item representation scheme: it rst

maps item text into a vector of discrete indices (called item code),

and then employs these indices to lookup the code embedding table

for deriving item representations. Such a scheme can be denoted

as “text

=⇒

code

=⇒

representation”. Based on this representation

scheme, we further propose an enhanced contrastive pre-training

approach, using semi-synthetic and mixed-domain code representa-

tions as hard negatives. Furthermore, we design a new cross-domain

ne-tuning method based on a dierentiable permutation-based net-

work. Extensive experiments conducted on six public benchmarks

demonstrate the eectiveness of the proposed approach, in both

cross-domain and cross-platform settings. Code and pre-trained

model are available at: https://github.com/RUCAIBox/VQ-Rec.

CCS CONCEPTS

•Information systems →Recommender systems.

†Beijing Key Laboratory of Big Data Management and Analysis Methods.

BCorresponding author.

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than the

author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or

republish, to post on servers or to redistribute to lists, requires prior specic permission

and/or a fee. Request permissions from permissions@acm.org.

WWW ’23, May 1–5, 2023, Austin, TX, USA

ACM ISBN 978-1-4503-9416-1/23/04. . . $15.00

https://doi.org/10.1145/3543507.3583434

ACM Reference Format:

Yupeng Hou, Zhankui He, Julian McAuley, Wayne Xin Zhao. 2023. Learning

Vector-Quantized Item Representation for Transferable Sequential Rec-

ommenders. In Proceedings of the ACM Web Conference 2023 (WWW ’23),

May 1–5, 2023, Austin, TX, USA. ACM, New York, NY, USA, 10 pages.

https://doi.org/10.1145/3543507.3583434

1 INTRODUCTION

Sequential recommender systems have been widely deployed on

various application platforms for recommending items of interest

to users. Typically, such a recommendation task is formulated as

a sequence prediction problem [

], inferring the next

item(s) that a user is likely to interact with based on her/his histori-

cal interaction sequences. Although a similar task formulation has

been taken for dierent sequential recommenders, it is dicult to

reuse an existing well-trained recommender for new recommenda-

tion scenarios [

]. For example, when a new domain emerges

with specic interaction characteristics, one may need to train a rec-

ommender from scratch, which is time-consuming and can suer

from cold-start issues. Thus, it is desirable to develop transferable

sequential recommenders [

], which can quickly adapt to

new domains or scenarios.

For this purpose, in recommender systems literature, early stud-

ies mainly conduct cross-domain recommendation methods [

] by transferring the learned knowledge from existing domains

to a new one. These studies mainly assume that shared information

(e.g., overlapping users/items [

] or common features [

])

are available for learning cross-domain mapping relations. How-

ever, in real applications, users or items are only partially shared or

completely non-overlapping across dierent domains (especially in

a cross-platform setting), making it dicult to eectively conduct

cross-domain transfer. Besides, previous content-based transfer

methods [

] usually design specic approaches tailored for

the data format of shared features, which is not generally useful in

various recommendation scenarios.

As a recent approach, several studies [

] propose to

leverage the generality of natural language texts (i.e., title and de-

scription text of items, called item text) for bridging the domain gap

in recommender systems. The basic idea is to employ the learned

text encodings via pre-trained language models (PLM) [

] as

arXiv:2210.12316v2 [cs.IR] 12 Feb 2023

WWW ’23, May 1–5, 2023, Austin, TX, USA Hou, et al.

universal item representations. Based on such item representations,

sequential recommenders pre-trained on the interaction data from

a mixture of multiple domains [

] have shown promising

transferability. Such a paradigm can be denoted as “text

=⇒

rep-

resentation”. Despite the eectiveness, we argue that the binding

between item text and item representations is “too tight” in previous

approaches [

], thus leading to two potential issues. First, since

these methods employ text encodings to derive item representations

(without using item IDs), text semantics have a direct inuence on

the recommendation model. Thus, the recommender might over-

emphasize the eect of text features (e.g., generating very similar

recommendations in texts) instead of sequential characteristics re-

ected in interaction data. Secondly, text encodings from dierent

domains (with varied distributions and semantics [

]) are not

naturally aligned in a unied semantic space, and the domain gap

existing in text encodings is likely to cause a performance drop

during multi-domain pre-training. The tight binding between text

encodings and item representations might exaggerate the negative

impact of the domain gap.

Considering these issues, our solution is to incorporate inter-

mediate discrete item indices (called codes in this work) in item

representation scheme and relax the strong binding between item

text and item representations, which can be denoted as “text

=⇒

code

=⇒

representation”. Instead of directly mapping text encodings

into item representations, we consider a two-step item representa-

tion scheme. Given an item, it rst maps the item text to a vector

of discrete indices (i.e., item code), and then aggregates the cor-

responding embeddings according to the item code as the item

representation. The merits of such a representation scheme are

twofold. Firstly, item text is mainly utilized to generate discrete

codes, which can reduce its inuence on the recommendation model

meanwhile inject useful text semantics. Second, the two mapping

steps can be learned or tuned according to downstream domains

or tasks, making it more exible to t new recommendation sce-

narios. To develop our approach, we highlight two key challenges

to address: (i) how to learn discrete item codes that are suciently

distinguishable for accurate recommendation; (ii) how to eectively

pre-train and adapt the item representations considering the varied

distribution and semantics across dierent domains.

To this end, we propose

VQ-Rec

, a novel approach to learn

ector-

uantized item representations for transferable sequential

Rec

ommenders. Dierent from existing transferable recommenders

based on PLM encoding, VQ-Rec maps each item into a discrete

𝐷

-dimensional code as the indices for embedding lookup. To ob-

tain semantically-rich and distinguishable item codes, we utilize

optimized product quantization (OPQ) techniques to discretize text

encodings of items. In this way, the discrete codes that preserve

the textual semantics are distributed over the item set in a more

uniform way, so as to be highly distinguishable. Since our repre-

sentation scheme does not modify the underlying backbone (i.e.,

Transformer), it is generally applicable to various sequential ar-

chitectures. To capture transferable patterns based on item codes,

we pre-train the recommender on a mixture of multiple domains

in a contrastive learning approach. Both mixed-domain and semi-

synthetic code representations are used as hard negatives to en-

hance the contrastive training. To transfer the pre-trained model

to a downstream domain, we propose a dierentiable permutation-

based network to learn the code-embedding alignment, and fur-

ther update the code embedding table to t the new domain. Such

ne-tuning is highly parameter-ecient, as only the parameters

involved in item representations need to be tuned.

Empirically, we conduct extensive experiments on six bench-

marks, including both cross-domain and cross-platform scenarios.

Experimental results demonstrate the strong transferability of our

approach. Especially, inductive recommenders purely based on item

text can recommend new items without re-training, and meanwhile

gain better performance on known items.

2 METHODOLOGY

In this section, we present the proposed transferable sequential

Rec

ommendation approach based on

ector-

uantized item in-

dices, named VQ-Rec.

2.1 Approach Overview

Task formulation.

We consider the sequential recommendation

task setting that multi-domain interaction data is available as train-

ing (or pre-training) data. Formally, the interaction data of a user

in some domain can be denoted as an interaction sequence

𝑠=

{𝑖1, 𝑖2, . . . , 𝑖𝑛}

(in chronological order), where each interacted item

𝑖

is associated with a unique item ID and text data, e.g., title or de-

scription (item text). Since a user is likely to interact with items from

multiple domains, we can derive multiple interaction sequences

for a user. Considering the large semantic gap across dierent do-

mains [

], we don’t combine the multiple interaction sequences of

a user into a single sequence, but instead keep these sequences per

domain. Note that the item IDs are not explicitly utilized to generate

item representations in our approach. The task goal is to pre-train

a transferable sequential recommender that can eectively adapt

to new domains (unseen in training data).

Solution overview.

To develop the sequential recommender, we

adopt the popular Transformer architecture [

] as the backbone

of our approach. It is built on the self-attention mechanism, tak-

ing as input item embeddings and positional embeddings at each

time step. Unlike previous related studies [

], we don’t include

any additional components (e.g., adaptors) into the Transformer

architecture, but instead learn transferable item representations

for feeding the backbone. The key novelty of our approach lies in

the new item representation scheme for sequential recommenders.

In this scheme, we rst map item text into a vector of discrete in-

dices (called an item code), and then employ these indices to lookup

the code embedding table for deriving item representations. Such a

scheme can be denoted as “text

=⇒

code

=⇒

representation”, which

removes the tight binding between item text and item represen-

tations. In order to learn and transfer such item representations,

we further propose specic strategies for contrastive recommender

pre-training and cross-domain recommender ne-tuning.

The overall framework of the proposed approach VQ-Rec is

depicted in Figure 1. We consider three key components for devel-

oping transferable recommenders: (i) how to represent the items

with vector-quantized code representation (Section 2.2); (ii) how to

train the recommenders based on the new representation scheme

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LearningVector-QuantizedItemRepresentationforTransferableSequentialRecommendersYupengHouhouyupeng@ruc.edu.cnGaolingSchoolofArtificialIntelligenceRenminUniversityofChinaBeijing,ChinaZhankuiHezhh004@eng.ucsd.eduUCSanDiegoSanDiego,California,USAJulianMcAuleyjmcauley@eng.ucsd.eduUCSanDiegoSanDiego,Calif...

展开>> 收起<<

Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: