Augmentations in Hypergraph Contrastive Learning Fabricated and Generative Tianxin Wei1 Yuning You2 Tianlong Chen3 Yang Shen2 Jingrui He1 Zhangyang Wang3

2025-05-02 0 0 819.56KB 14 页 10玖币

侵权投诉

Augmentations in Hypergraph Contrastive Learning:

Fabricated and Generative

Tianxin Wei1∗, Yuning You2∗, Tianlong Chen3, Yang Shen2, Jingrui He1, Zhangyang Wang3

1University of Illinois Urbana-Champaign, 2Texas A&M University, 3University of Texas at Austin

{twei10,jingrui}@illinois.edu,{yuning.you,yshen}@tamu.edu,

{tianlong.chen,atlaswang}@utexas.edu

Abstract

This paper targets at improving the generalizability of hypergraph neural networks

in the low-label regime, through applying the contrastive learning approach from

images/graphs (we refer to it as

HyperGCL

). We focus on the following question:

How to construct contrastive views for hypergraphs via augmentations? We pro-

vide the solutions in two folds. First, guided by domain knowledge, we

fabricate

two schemes to augment hyperedges with higher-order relations encoded, and

adopt three vertex augmentation strategies from graph-structured data. Second,

in search of more effective views in a data-driven manner, we for the ﬁrst time

propose a hypergraph generative model to

generate

augmented views, and then

an end-to-end differentiable pipeline to jointly learn hypergraph augmentations

and model parameters. Our technical innovations are reﬂected in designing both

fabricated and generative augmentations of hypergraphs. The experimental ﬁndings

include: (i) Among fabricated augmentations in HyperGCL, augmenting hyper-

edges provides the most numerical gains, implying that higher-order information

in structures is usually more downstream-relevant; (ii) Generative augmentations

do better in preserving higher-order information to further beneﬁt generalizability;

(iii) HyperGCL also boosts robustness and fairness in hypergraph representation

learning. Codes are released at

https://github.com/weitianxin/HyperGCL

1 Introduction

Hypergraphs have raised a surge of interests in the research community [

] due to their innate

capability of capturing higher-order relations [

]. They offer a powerful tool to model complicated

topological structures in broad applications, e.g., recommender systems [

], ﬁnancial analyses

[

], and bioinformatics [

]. Concomitant with the trend, hypergraph neural networks

(HyperGNNs) have recently been developed [1, 2, 3] for hypergraph representation learning.

This paper focuses on the few-shot scenarios of hypergraphs, i.e., task-speciﬁc labels are scarce,

which are ubiquitous in real-world applications of hypergraphs [

] and empirically restrict the

generalizability of HyperGNNs. Inspired by the emerging self-supervised learning on images/graphs

[

], especially the contrastive approaches [

25], we set out to leverage contrastive self-supervision to address the problem.

Nevertheless, one challenge stands out: How to build contrastive views for hypergraphs? The success

of contrastive learning hinges on the appropriate view construction, otherwise it would result in

“negative transfer” [

]. However, it is non-trivial to build hypergraph views due to their overly

intricate topology, i.e., there are

e=1N

e

possibilities for one hyperedge on

vertices, versus

N

2

for one edge in graphs. To date, the only way of contrasting is between the representations

*Equal contribution.

36th Conference on Neural Information Processing Systems (NeurIPS 2022).

arXiv:2210.03801v1 [cs.LG] 7 Oct 2022

of hypergraphs and their clique-expansion graphs [

], which is computationally expensive as

multiple neural networks of different modalities (hypergraphs and variants of expanded graphs) need

to be optimized. More importantly, contrasting on clique expansion has the risk of losing higher-order

information via pulling representations of hypergraphs and graphs close.

Contributions.

Motivated by [

] that appropriate data augmentations sufﬁce for the effective

contrastive views, and intuitively they are more capable of preserving higher-order relations in

hypergraphs compared to clique expansion, we explore on the question in this paper, how to design

augmented views of hypergraphs in contrastive learning (

HyperGCL

). Our answers are in two folds.

We ﬁrst assay whether

fabricated

augmentations guided by domain knowledge are suited for Hy-

perGCL. Since hypergraphs are composed of hyperedges and vertices, to augment hyperedges, we

propose two strategies that (i) directly perturb on hyperedges, and (ii) perturb on the “edges” between

hyperedges and vertices in the converted bipartite graph; To augment vertices, we adopt three schemes

of vertex dropping, attribute masking and subgraph from graph-structured data [

]. Our ﬁnding is

that, different from the fact that vertex augmentations beneﬁt more on graphs, hypergraphs mostly ben-

eﬁt from hyperedge augmentations (up to 9% improvement), revealing that higher-order information

encoded in hyperedges is usually more downstream-relevant (than information in vertices).

Furthermore, in search of even better augmented views but in a data-driven manner, we study

whether/how augmentations of hypergraphs could be learned during contrastive learning. To this

end, for the ﬁrst time, we propose a novel variational hypergraph auto-encoder architecture, as a

hypergraph

generative

model, to parameterize a certain augmentation space of hypergraphs. In addi-

tion, we propose an end-to-end differentiable pipeline utilizing Gumbel-Softmax [

], to jointly learn

hypergraph augmentations and model parameters. Our observation is that generative augmentations

can better capture the higher-order information and achieve state-of-the-art performance on most of

the benchmark data sets (up to 20% improvement).

The aforementioned empirical evidences (for generalizability) are drawn from comprehensive experi-

ments on 13 datasets. Moreover, we introduce the robustness and fairness evaluation for hypergraphs,

and show that HyperGCL in addition boosts robustness against adversarial attacks and imposes

fairness with regard to sensitive attributes.

The rest of the paper is organized as follows. We discuss the related work in Section 2, introduce

HyperGCL in Section 3, present the experimental results in Section 4, and conclude in Section 5.

2 Related Work

Hypergraph neural networks.

Hypergraphs, which are able to encode higher-order relationships,

have attracted signiﬁcant attentions in recent years. In the machine learning community, hypergraph

neural networks are developed for effective hypergraph representations. HGNN [

] adopt the clique

expansion technique and designs the weighted hypergraph Laplacian for message passing. HyperGCN

[

] proposes the generalized hypergraph Laplacian and explores adding the hyperedge information

through mediators. The attention mechanism [

] is also designed to learn the importance within

hypergraphs. However, the expanded graph will inevitably cause distortion and lead to unsatisfactory

performance. There is also another line of works such as UniGNN [

] and HyperSAGE [

] which

try to perform message passing directly on the hypergraph to avoid the information loss. A recent

work [

] provides an AllSet framework to unify the existing studies with high expressive power and

achieves state-of-the-art performance on comprehensive benchmarks. The work utilizes deep multiset

functions [33] to identify the propagation and aggregation rules in a data-driven manner.

Contrastive self-supervised learning.

Contrastive self-supervision [

] has achieved un-

precedented success in computer vision. The core idea is to learn an embedding space where samples

from the same instance are pulled closer and samples from different instances are pushed apart. Re-

cent works start to cross-pollinate between contrastive learning and graph neural networks to for more

generalizable graph representations. Typically, they design some fabricated augmentations guided by

domain knowledge, such as edge perturbation, feature masking or vertex dropping, etc. Nevertheless,

contrastive learning on hypergraphs remains largely unexplored. Most existing works [

]

design pretext tasks for hypergraphs and mainly focus on recommender systems [

], via

contrasting between graphs and hypergraphs which might lose important higher-order information.

In this work, we explore on the structure of hypergraph itself to construct contrastive views.

Figure 1: The framework of hypergraph contrastive learning (HyperGCL). The ellipses represent the

hyperedges. Two contrastive views are generated by hypergraph augmentations

and

from the

augmentation collection

f(·)

and

h(·)

are shared encoder and projection head respectively. In the

ﬁgure, we show two examples of hypergraph augmentations. At the top, the dotted ellipse denotes

the deleted hyperedge. At the bottom, one vertex in the dotted hyperedge is removed.

3 Methods

3.1 Hypergraph Contrastive Learning

A hypergraph is denoted as

G={V,E} ∈ G

where

V={v1, ..., v|V|}

is the set of vertices and

E={e1, ..., e|E|}

is the set of hyperedges. Each hyperedge

en={v1, ..., v|en|}

represents the

higher-order interaction among a set of vertices. State-of-the-art approaches to encode such complex

structures are hypergraph neural networks (HyperGNNs) [

], mapping the hypergraph to a

D-dimension latent space via f:G→RDwith higher-order message passing.

Motivated from learning on images/graphs, we adopt contrastive learning to further improve the

generalizability of HyperGNNs in the low-label regime (HyperGCL). Main components of our Hy-

perGCL, similar to images/graphs [

] include: (i)

hypergraph augmentations for contrastive

views

, (ii) HyperGNNs as hypergraph encoders, (iii) projection head

h(·)

for representations, and (iv)

contrastive loss for optimization. The overall pipeline is shown in Figure 1. Detailed descriptions and

training procedure are shown in Appendix B. The main challenge here is how to effectively augment

hypergraphs to build contrastive views.

3.2 Fabricated Augmentations for Hypergraphs

Vertex

Hyperedge

Transform

Figure 2: Conversion from hypergraph to equivalent bipartite graph.

We ﬁrst explore whether

manually designed augmen-

tations are suited for Hyper-

GCL. Since hyperedges and

vertices compose a hyper-

graph, augmentations are

fabricated with regards to

topology and node features,

respectively.

A1. Perturbing hyper-

edges.

The most direct aug-

mentation on higher-order

interactions is to perturb on

the set of hyperedges. Since adding a hyperedge is confronted with the combinatorial challenge (see

Sec. 1 of introduction), here we focus on randomly removing the existing hyperedges following an

i.i.d. Bernoulli distribution. The underlying assumption is that the partially missing higher-order

relations do not signiﬁcantly affect the semantic meaning of hypergraphs.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AugmentationsinHypergraphContrastiveLearning:FabricatedandGenerativeTianxinWei1,YuningYou2,TianlongChen3,YangShen2,JingruiHe1,ZhangyangWang31UniversityofIllinoisUrbana-Champaign,2TexasA&MUniversity,3UniversityofTexasatAustin{twei10,jingrui}@illinois.edu,{yuning.you,yshen}@tamu.edu,{tianlong.chen,a...

展开>> 收起<<

Augmentations in Hypergraph Contrastive Learning Fabricated and Generative Tianxin Wei1 Yuning You2 Tianlong Chen3 Yang Shen2 Jingrui He1 Zhangyang Wang3.pdf

共14页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Augmentations in Hypergraph Contrastive Learning Fabricated and Generative Tianxin Wei1 Yuning You2 Tianlong Chen3 Yang Shen2 Jingrui He1 Zhangyang Wang3

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: