Meta-node A Concise Approach to Effectively Learn Complex Relationships in Heterogeneous Graphs Jiwoong Park1Jisu Jeong2Kyungmin Kim2Jin Young Choi1

2025-05-02 0 0 3.59MB 9 页 10玖币

侵权投诉

Meta-node: A Concise Approach to

Effectively Learn Complex Relationships in Heterogeneous Graphs

Jiwoong Park 1Jisu Jeong 2Kyungmin Kim 2Jin Young Choi 1

1Seoul National University, 2NAVER

Abstract

Existing message passing neural networks for heterogeneous

graphs rely on the concepts of meta-paths or meta-graphs due

to the intrinsic nature of heterogeneous graphs. However, the

meta-paths and meta-graphs need to be pre-conﬁgured before

learning and are highly dependent on expert knowledge to

construct them. To tackle this challenge, we propose a novel

concept of meta-node for message passing that can learn

enriched relational knowledge from complex heterogeneous

graphs without any meta-paths and meta-graphs by explicitly

modeling the relations among the same type of nodes. Un-

like meta-paths and meta-graphs, meta-nodes do not require

any pre-processing steps that require expert knowledge. Go-

ing one step further, we propose a meta-node message pass-

ing scheme and apply our method to a contrastive learning

model. In the experiments on node clustering and classiﬁca-

tion tasks, the proposed meta-node message passing method

outperforms state-of-the-arts that depend on meta-paths. Our

results demonstrate that effective heterogeneous graph learn-

ing is possible without the need for meta-paths that are fre-

quently used in this ﬁeld.

Introduction

Graph Neural Networks (GNNs) (Gori, Monfardini, and

Scarselli 2005; Gilmer et al. 2017; Kipf and Welling 2017;

Hamilton, Ying, and Leskovec 2017; Veliˇ

ckovi´

c et al. 2018;

Xu et al. 2018) have become the de facto standard for rep-

resentation learning on graph-structured data. Among the

differing architectures for GNNs, Message Passing Neu-

ral Networks (MPNNs) (Gilmer et al. 2017; Morris et al.

2019) in which nodes exchange messages (i.e., representa-

tions) along edges, are considered well-known and effective

mechanisms. Since GNNs were ﬁrst proposed (Gori, Mon-

fardini, and Scarselli 2005), the majority of efforts in this

ﬁeld have been aimed at learning representations for homo-

geneous graphs with a single type of nodes and a single type

of edges (i.e., relationships).

However, graph-structured datasets in real-world applica-

tions are not limited to a single type of nodes and edges.

For instance, in the movie network of Figure 1 (a), there ex-

ist multiple types of nodes (movies, actors, directors, and

producers) and multiple types of edges (acting, ﬁlming, and

producing). This kind of graph that has multiple types of

nodes and edges is called heterogeneous graph (Wang et al.

Movie

Actor

Director Producer

(a)

(b)

Meta-node

Movie Director Movie

Actor Movie Actor

Movie Producer

Movie

ActorActor

Figure 1: (a) An example of heterogeneous movie networks.

There exists four types of nodes: movie, actor, director, and

producer. (b) The proposed meta-node: each meta-node ag-

gregates messages of all nodes of each type and returns the

aggregated message to each node when passing messages

to the next layer. (c) Example of two meta-paths which are

compositions of different types of nodes. (d) A meta-graph

which is a composition of multiple meta-paths.

2020; Yang et al. 2020). To capture complex relations in het-

erogeneous graphs, the representation learning model must

consider the distinct nature of multiple types of nodes and

edges. Thus simply plugging heterogeneous graphs into con-

ventional MPNNs is inadequate because the MPNNs can-

not distinguish multiple node and edge types. To deal with

this problem, recently, Heterogeneous Graph Neural Net-

works (HGNNs) (Schlichtkrull et al. 2018; Zhang et al.

2019; Wang et al. 2019; Kim et al. 2019; Yun et al. 2019;

Wang et al. 2021; Ren et al. 2020; Hu et al. 2020) have been

proposed to extract useful knowledge from heterogeneous

graphs by leveraging the power of GNNs.

To learn complex relational knowledge from heteroge-

neous graphs, most HGNNs rely on the node composi-

tions constructed before training. This dependency on pre-

processing steps of HGNNs is from the unique character-

istics of heterogeneous graphs. As an example in Figure 1,

most of the heterogeneous graphs are k-partite graphs whose

nodes can be divided into kindependent sets. Due to the na-

ture of k-partite graphs, all that is given are sparse inter-type

arXiv:2210.14480v1 [cs.LG] 26 Oct 2022

Table 1: Predeﬁned meta-paths of real-world datasets. In this table, it can be noticed that most of Rare inter-type relations and

Ptarget on intra-type relations by setting the same type of nodes at both ends of P.

Dataset A R P

DBLP A, P, T, C A-P, P-T, P-C APA, APCPA, APTPA

IMDB M, D, A M-D, M-A MDM, MAM

ACM P, A, S P-A, P-S PAP, PSP

AMiner P, A, R P-A, P-R PAP, PRP

Freebase M, D, A, P M-D, M-A, M-P MAM, MDM, MPM

Last.FM U, A, T U-U, U-A, A-T UU, UAU, UATAU, AUA, AUUA, ATA

Yelp U, B, Co, Ci, Ca U-U, U-B, U-Co, B-Ci, B-Ca UBU, UCoU, UBCiBU, UBCaBU, BUB, BCiB, BCaB, BUCoUB

Douban U, M, G, L, D, A, T U-U, U-G, U-M, U-L, M-D, M-T, M-A MUM, MTM, MDM, MAM, UMU, UMAMU, UMDMU, UMTMU

relations (i.e., edges between different types of nodes). How-

ever, using only these inter-type relations is not enough to

extract useful knowledge from the intricate relations in the

data. To resolve this problem, most HGNNs rely on addi-

tional predeﬁned relational information, and the most com-

monly used methods are meta-path (Sun et al. 2011) and

meta-graph (Fang et al. 2016; Huang et al. 2016), each of

which are a composition of different types of nodes and mul-

tiple meta-paths as shown in Figure 1 (c) and (d). As we will

show later, nearly all meta-paths implicitly derive intra-type

relations (i.e., relations between the same type of nodes) by

manipulating given inter-type relations.

However, there exist three major problems with using pre-

deﬁned methods such as meta-paths for heterogeneous graph

learning. Firstly, there exist certain limitations on induc-

ing intra-type relations from predeﬁned inter-type relations.

When the given inter-type relations are sparse or noisy, in-

duced intra-type relations can also be affected. Secondly,

the appropriate composition of nodes and edges (design-

ing meta-paths and meta-graphs) for representation learning

requires signiﬁcant domain-speciﬁc knowledge. Thus, it is

extremely hard to know which combinations of nodes and

edges are suitable for learning useful representations, espe-

cially in unsupervised environments. Lastly, although there

exist attempts to learn appropriate meta-paths beyond given

ones (Yun et al. 2019), several multiplications of the adja-

cency matrix are required. Due to the high computational

cost of multiple matrix multiplications, their method is lim-

ited to very small datasets (Lv et al. 2021).

To circumvent the above limitations of current meth-

ods, we propose a novel concept of meta-node to construct

simple and powerful MPNNs for learning heterogeneous

graphs. Meta-nodes are virtual nodes in which one meta-

node is added to the graph for each type of node in the het-

erogeneous graph. Each meta-node is connected to all nodes

of each type as illustrated in Figure 1 (b). By introducing

meta-nodes, message passing is no longer limited to sparse

inter-type relations, and every node can directly perform

message passing with other nodes of the same type via meta-

nodes. To do so, we can enrich the information on the rela-

tionship by adding explicit intra-type relations to the given

inter-type relations. After introducing the concept of meta-

nodes, we propose a message passing scheme via meta-node

to learn both intra- and inter-type relations effectively.

Unsupervised representation learning on heterogeneous

graphs has become one of the major challenges in graph-

structured data learning, as it can pave the way to make

use of large amounts of unlabeled multi-modal data. Thus,

we validate the proposed message passing scheme by ap-

plying it to unsupervised representation learning for graph-

structured data. To do so, we apply our meta-node mes-

sage passing layer to the encoder of Deep Graph Infomax

(Veliˇ

ckovi´

c et al. 2019) which is one of the most well-

known graph contrastive models. Through downstream tasks

on four real-world heterogeneous graph datasets, we vali-

date the proposed message passing scheme. We conﬁrm that

our meta-node message passing layer learns rich relational

information and shows competitive performance compared

to existing state-of-the-art HGNNs even without any meta-

paths.

Related Work

Meta-path. A meta-path (Sun et al. 2011) Pis deﬁned as

a path that has a form of A1

−−→ A2

−−→ · · · Rl

−→ Al+1 (ab-

breviated as A1A2· · · Al+1) which describes relations be-

tween A1and Al+1 ∈ A with a composition of relations

R1, R2, . . . , Rl∈ R, where Aand Rdenote sets of node

types and edge types of heterogeneous graphs, respectively.

Each meta-path can describe a semantic relation between

nodes at both ends of the meta-path. For instance, in Figure

1 (c), the meta-path of movie-director-movie can describe

the relationship between two movies by which the director

ﬁlmed them. Nearly all meta-paths of the real-world datasets

(Wang et al. 2019; Fu et al. 2020; Wang et al. 2020, 2021)

are implicitly composed for intra-type relations by setting

the same type of nodes at both ends of Pusing given inter-

type relations Ras shown in Table 1.

Representation Learning for Heterogeneous Graphs.

For several past years, there have been many efforts to learn

representations of heterogeneous graphs based on random-

walk-based methods (Dong, Chawla, and Swami 2017; Fu,

Lee, and Lei 2017; Jeong et al. 2020; He et al. 2019) or

GNNs methods (Schlichtkrull et al. 2018; Shi et al. 2018;

Zhang et al. 2019; Yun et al. 2019; Wang et al. 2019; Zhao

et al. 2020; Fu et al. 2020; Hu et al. 2020; Zhao et al. 2021).

Nowadays, HGNNs leveraging the power of GNNs show a

remarkable ability to learn intricate relations between multi-

ple types of nodes and edges both in semi-supervised and

unsupervised conditions. For instance, in semi-supervised

learning, HAN (Zhang et al. 2019) proposed attention-based

MPNNs using meta-paths to take each semantic meaning of

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Meta-node:AConciseApproachtoEffectivelyLearnComplexRelationshipsinHeterogeneousGraphsJiwoongPark1JisuJeong2KyungminKim2JinYoungChoi11SeoulNationalUniversity,2NAVERAbstractExistingmessagepassingneuralnetworksforheterogeneousgraphsrelyontheconceptsofmeta-pathsormeta-graphsduetotheintrinsicnatureofhete...

展开>> 收起<<

Meta-node A Concise Approach to Effectively Learn Complex Relationships in Heterogeneous Graphs Jiwoong Park1Jisu Jeong2Kyungmin Kim2Jin Young Choi1.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Meta-node A Concise Approach to Effectively Learn Complex Relationships in Heterogeneous Graphs Jiwoong Park1Jisu Jeong2Kyungmin Kim2Jin Young Choi1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: