DIGAT Modeling News Recommendation with Dual-Graph Interaction Zhiming Mao12 Jian Li3 Hongru Wang12 Xingshan Zeng4 Kam-Fai Wong12 1The Chinese University of Hong Kong Hong Kong China

2025-04-27 0 0 3.65MB 13 页 10玖币

侵权投诉

DIGAT: Modeling News Recommendation with Dual-Graph Interaction

Zhiming Mao1,2, Jian Li3, Hongru Wang1,2, Xingshan Zeng4, Kam-Fai Wong1,2

1The Chinese University of Hong Kong, Hong Kong, China

2MoE Key Laboratory of High Conﬁdence Software Technologies, China

3Tencent, Shenzhen, China

1,2{zmmao,hrwang,kfwong}@se.cuhk.edu.hk

3lijianjack@gmail.com 4zxshamson@gmail.com

Abstract

News recommendation (NR) is essential for

online news services. Existing NR meth-

ods typically adopt a news-user representation

learning framework, facing two potential lim-

itations. First, in news encoder, single can-

didate news encoding suffers from an insufﬁ-

cient semantic information problem. Second,

existing graph-based NR methods are promis-

ing but lack effective news-user feature inter-

action, rendering the graph-based recommen-

dation suboptimal. To overcome these limi-

tations, we propose dual-interactive graph at-

tention networks (DIGAT) consisting of news-

and user-graph channels. In the news-graph

channel, we enrich the semantics of single can-

didate news by incorporating the semantically

relevant news information with a semantic-

augmented graph (SAG). In the user-graph

channel, multi-level user interests are repre-

sented in a news-topic graph. Most notably,

we design a dual-graph interaction process to

perform effective feature interaction between

the news and user graphs, which facilitates

accurate news-user representation matching.

Experiment results on the benchmark dataset

MIND show that DIGAT outperforms existing

news recommendation methods1. Further ab-

lation studies and analyses validate the effec-

tiveness of (i) semantic-augmented news graph

modeling and (ii) dual-graph interaction.

1 Introduction

News recommendation is an important technique to

provide people with the news which satisﬁes their

personalized reading interests (Okura et al.,2017;

Wu et al.,2020). Effective news recommender

systems require both accurate textual modeling on

news content (Wang et al.,2018;Wu et al.,2019d;

Wang et al.,2020) and personal-interest modeling

on user behavior (Hu et al.,2020b;Qi et al.,2021c).

Hence, most news recommendation methods (An

Our code is available at https://github.com/Veason-

silverbullet/DIGAT.

News Recommender

Model

News Encoder

User Encoder

Click Predictor

Single candidate news

User historical clicked news

Click Score

News representation

User representation

Figure 1: The typical news-user representation learning

framework for news recommendation.

et al.,2019;Wu et al.,2019a,b,c,d;Ge et al.,2020;

Qi et al.,2021b,c) adopt a news-user representation

learning framework to learn discriminative news

and user representations, as illustrated in Figure 1.

Though promising, there are still two potential

limitations in the existing news recommendation

framework. First, in news encoder, single candidate

news encoding suffers from an insufﬁcient seman-

tic information problem. Unlike long-term items in

common recommendation (e.g., E-commerce prod-

uct recommendation), the candidate news items are

short-term and suffer from the cold start problem.

In the real-world setting, news recommender sys-

tems usually handle the latest news, where existing

user-click interactions are always not available

Hence, it is intractable to use existing user-click

records to enrich the information of candidate news.

On the other hand, compared to abundant histor-

ical clicked news in user encoder, the single can-

didate news may not contain sufﬁcient semantic

information for accurate news-user representation

matching in the click prediction stage. Prior studies

(Wu et al.,2019a,c;Qi et al.,2021c) pointed out

that users were usually interested in speciﬁc news

topics (e.g., Sports). Empirically, the text of single

candidate news does not contain enough syntactic

and semantic information to accurately represent a

genre of news topic and match user interests.

From the viewpoint of experimental dataset, most candi-

date news in test data does not appear in training user history.

arXiv:2210.05196v2 [cs.CL] 14 Oct 2022

Second, previous studies generally follow two

research directions to model user history, i.e., se-

quence and graph modeling. Formulating user his-

tory as a sequence of user’s clicked news is a more

prevalent direction, based on which time-sequential

models (Okura et al.,2017;An et al.,2019;Qi et al.,

2021b) and attentive models (Zhu et al.,2019;Wu

et al.,2019a,b,d;Qi et al.,2021a,c) are proposed.

Besides, graph modeling is proved effective for rec-

ommender systems (Chen et al.,2020). Ge et al.

(2020) and Hu et al. (2020b) formulate news and

users jointly in a bipartite graph to model news-

user interaction. However, since most candidate

news in test data has no existing interaction with

users (i.e., cold-news), the isolated cold-news nodes

cause this bipartite graph modeling degenerate. Re-

cent works formulate user history as heterogeneous

graphs and employ advanced graph learning meth-

ods to extract the user-graph representations (Hu

et al.,2020a;Mao et al.,2021;Wu et al.,2021).

These works focus on how to extract ﬁne-grained

representations from the user-graph side but neglect

necessary feature interaction between the candidate

news and user-graphs.

In this work, we propose

ual-

nteractive

raph

tention networks (DIGAT) to address the afore-

mentioned limitations. DIGAT consists of news-

and user-graph channels to encode the candidate

news and user history, respectively. In the news-

graph channel, we introduce semantic-augmented

graph (SAG) modeling to enrich the semantic repre-

sentation of the single candidate news. In SAG, the

original candidate news is regarded as the root node,

while the semantic-relevant news documents are

represented as the extended nodes to augment the

semantics of the candidate news. We integrate the

local and global contexts of SAG as the semantic-

augmented candidate news representations.

In the user-graph channel, motivated by Mao

et al. (2021) and Wu et al. (2021), we model user

history with a news-topic graph to represent multi-

levels of user interests. Most notably, we design a

dual-graph interaction process to learn news- and

user-graph representations with effective feature

interaction. Different from the individual graph

attention network (Veliˇ

ckovi´

c et al.,2018), DIGAT

updates news and user graph embeddings with the

interactive attention mechanism. Particularly, in

each layer of the dual-graph, the user (news) graph

context is incorporated into its dual news (user)

node embedding learning iteratively.

Extensive experiments on the benchmark dataset

MIND (Wu et al.,2020) show that DIGAT signiﬁ-

cantly outperforms the existing news recommenda-

tion methods. Further ablation studies and analyses

conﬁrm that semantic-augmented news graph mod-

eling and dual-graph interaction can substantially

improve news recommendation performance.

2 Related Work

Personalized news recommendation is important to

online news services (Okura et al.,2017;Yi et al.,

2021). Existing neural news recommendation meth-

ods typically aim to learn informative news and

user representations (Wang et al.,2018;Zhu et al.,

2019;An et al.,2019;Wu et al.,2019a,b,d;Liu

et al.,2020;Wang et al.,2020;Qi et al.,2021a,b,c;

Wu et al.,2021;Li et al.,2022). For example,

An et al. (2019) used a CNN network to extract

textual representations from news titles and used

a GRU network to learn short-term user interests

combined with long-term user embeddings. The

matching probabilities between candidate news and

users are computed over the learned news and user

representations. Wu et al. (2019d) utilized multi-

head self-attention networks to learn informative

news and user representations from news titles and

user clicked history. These methods regarded the

single candidate news as the input to news encoder,

which may not contain sufﬁcient semantics to rep-

resent a user-interested news topic. Different from

these methods, we encode the candidate news with

semantic-augmented graphs to enrich its semantic

representations. More recently, graph-based meth-

ods were proposed for news recommendation (Ge

et al.,2020;Hu et al.,2020a,b;Mao et al.,2021;

Wu et al.,2021). For example, Wu et al. (2021)

proposed a heterogeneous graph pooling method

to learn ﬁne-grained user representations. How-

ever, feature interaction between candidate news

and users is inadequate or neglected in these meth-

ods. In contrast, our approach models effective

feature interaction between news and user graphs

for accurate news-user representation matching.

3 Approach

Problem Formulation.

Denote the clicked-news

history of a user

Hu= [n1, n2, ..., n|H|]

, con-

taining

|H|

clicked news items. For the news

, its

textual content consists of a sequence of

|T|

words

Tn= [w1, w2, ..., w|T|]

. Based on

and

the goal of news recommendation is to predict the

score

ˆsn,u

, which indicates the probability of the

user

clicking the candidate news

ncan

. The rec-

ommendation result is generated by ranking the

user-click scores of multiple candidate news items.

3.1 News Semantic Representation

We introduce how to extract semantic representa-

tion from news content text

Tn= [w1, w2, ..., w|T|]

Our news encoder ﬁrst maps the news word to-

kens into word embeddings

En= [e1, e2, ..., e|T|]

Then, we use the multihead self-attention network

MSA(Q,K,V)

of Transformer encoder (Vaswani

et al.,2017) to learn the contextual representations

Hn∈R|T|×d

(where

is the feature dimension).

Finally, we employ an attention network

fatt(·)

aggregate the news semantic representation

h∈Rd

Hn=MSA(Q = En,K = En,V = En)

h=fattReLU(Hn)(1)

The attentive aggregation function

fatt(·)

is imple-

mented by a feed-forward network in our experi-

ments. It is worth noting that the semantic news

encoder in our framework is plug-and-play, which

can be easily replaced by any other textual encoders

or pretrained language models, e.g., BERT (Devlin

et al.,2019) or DeBERTa (He et al.,2021).

3.2 News Graph Encoding Channel

In this section, we will explain the news semantic-

augmented graph (SAG) construction and graph

context learning. Our motivation is to retrieve

semantic-relevant news from training corpus and

construct a semantic-augmented graph to enrich

the semantics of the original single candidate news.

3.2.1 News Graph Construction

Semantic-relevant News Retrieval.

Pretrained

language models (PLM) have achieved remarkable

performance (Reimers and Gurevych,2019,2020)

on semantic textual similarity (STS) benchmarks.

Motivated by Lewis et al. (2020), we utilize a PLM

φ(·)

to retrieve semantic-relevant news from train-

ing news corpus

to augment the semantic informa-

tion of the original single candidate news. In the

retrieval process, the semantic similarity score

si,j

of news

and

(corresponding texts

and

)

is computed by the similarity function sim(·,·):

si,j =simni, nj=cosineφ(Ti), φ(Tj)(2)

Speciﬁcally, we use pretrained mpnet-base-v2 (Song et al.,

2020) in the Sentence Transformers library https://www.sbert.

net/docs/pretrained_models.html as the news retrieval PLM.

Semantic-augmented Graph (SAG).

For the orig-

inal candidate news

ncan

, we initialize it as the root

node

of the semantic-augmented news graph

We build

by repeatedly extending semantic-

relevant neighboring nodes to existing nodes of

In each graph construction step, for an existing

node

(correspoding news

) of

news

documents

{Nj}M

j=1

are retrieved from the news

corpus

{NC}

with the highest semantic similarity

scores

{si,j }M

j=1

. We extend the nodes

{vj}M

j=1

neighboring nodes to the node

by adding bidi-

rectional edge

{ei,j }M

j=1

between them. To heuris-

tically discover semantic-relevant news in higher-

order relations, we repeatedly extend the semantic-

relevant news nodes within

hops from the root

node. The scale of news graph

is approximated

to be

O(MK)

. Detailed SAG construction and

qualitative analysis are provided in Appendix A.

3.2.2 News Graph Context Extraction

Given an SAG

generated from the candidate

news node

with

semantic-relevant news nodes

{vi}N

i=1

, we use the semantic news encoder (intro-

duced in Section 3.1) to extract their semantic rep-

resentations as hn,0∈Rdand {hn,i}N

i=1 ∈RN×d.

We aim to extract the graph context

cn∈Rd

which augments the semantics of the candidate

news

ncan

by aggregating the information of

We consider the original semantics of the candidate

news preserved in the root node

and regard the

local graph context as

n=hn,0∈Rd

. Besides,

we employ an attention module to aggregate the

global graph context

n∈Rd

from the semantic-

relevant news nodes to encode the overall semantic

information of

. In the attention module, we

regard the root node embedding

hn,0

as the query

and the semantic-relevant news node embeddings

{hn,i}N

i=1 as the key-value pairs:

ei=(hn,0WQ

n)(hn,iWK

n)T

√d

αi= softmax(ei) = exp(ei)

j=1 exp(ej)

i=1

αihn,i (3)

, where

n∈Rd×d

and

n∈Rd×d

are param-

eter matrices. We integrate the local and global

graph contexts by a simple feed-forward gating net-

work

FFNg(·)

to derive the news graph context

cn=FFNg[hL

n;hG

n](4)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DIGAT:ModelingNewsRecommendationwithDual-GraphInteractionZhimingMao1;2,JianLi3,HongruWang1;2,XingshanZeng4,Kam-FaiWong1;21TheChineseUniversityofHongKong,HongKong,China2MoEKeyLaboratoryofHighCondenceSoftwareTechnologies,China3Tencent,Shenzhen,China1;2{zmmao,hrwang,kfwong}@se.cuhk.edu.hk3lijianjack@g...

展开>> 收起<<

DIGAT Modeling News Recommendation with Dual-Graph Interaction Zhiming Mao12 Jian Li3 Hongru Wang12 Xingshan Zeng4 Kam-Fai Wong12 1The Chinese University of Hong Kong Hong Kong China.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

DIGAT Modeling News Recommendation with Dual-Graph Interaction Zhiming Mao12 Jian Li3 Hongru Wang12 Xingshan Zeng4 Kam-Fai Wong12 1The Chinese University of Hong Kong Hong Kong China

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: