Corresponding author Yawen Li warmly0716126.com. Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information

2025-04-27 0 0 569.26KB 5 页 10玖币

*Corresponding author: Yawen Li (warmly0716@126.com). 
Unsupervised Semantic Representation Learning of Scientific 
Literature Based on Graph Attention Mechanism and 
Maximum Mutual Information 
Hongrui Gao1, Yawen Li2*, Meiyu Liang1, Zeli Guan1   
1Beijing Key Laboratory of Intelligent Communication Software and Multimedia, School of Computer Science 
(National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876 
2School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing 100876
 
Abstract: Since  most  scientific  literature  data  are 
unlabeled, this makes unsupervised graph-based semantic 
representation  learning  crucial.  Therefore,  an 
unsupervised semantic representation learning method of 
scientific literature based on graph attention mechanism 
and maximum mutual information (GAMMI) is proposed. 
By  introducing  a  graph  attention  mechanism,  the 
weighted summation of nearby node features makes the 
weights of adjacent node features entirely depend on the 
node  features.  Depending  on the  features  of  the  nearby 
nodes, different weights can be applied to each node in 
the graph. In addition, an unsupervised graph contrastive 
learning  strategy  is  proposed  to  solve  the  problem  of 
being  unlabeled  and  scalable  on  large-scale  graphs.  By 
comparing  the mutual  information  between  the positive 
and  negative  local  node  representations  on  the  latent 
space  and  the  global  graph  representation,  the  graph 
neural  network  can  capture  both  local  and  global 
information.  Experimental  results  demonstrate 
competitive  performance  on  various  node  classification 
benchmarks, achieving good results and sometimes even 
surpassing the performance of supervised learning. 
Keywords:  Semantic  representation;  Graph  neural 
network; Graph attention; Maximum mutual information 
1 Introduction 
Currently, scientific literature resources are flooding the 
Internet  [1][2].  How  to  extract  important  information 
from  scientific  literature  and  effectively  represent  them 
semantically is the core issue to realize the classification, 
retrieval, and recommendation of scientific literature.   
Traditional methods mainly rely on expert experience to 
construct  artificial  features  to  represent  scientific 
literature. For example, in academic information retrieval, 
literature titles and abstract texts are used to construct an 
inverted index [3]. But every time new text data is added, 
it  is  tedious  to  rebuild  the  entire  index.  In  literature 
classification and recommendation systems, bag of words 
model  [4], vector space  model, and topic  model  [5] are 
used  to  build  scientific  literature  vectors.  However,  the 
shortcomings  of  the  bag  of  words  model  are  obvious. 
When  the  vocabulary  increases,  there  are  at  most  ten 
words used for each sentence, which leads to the sparse 
matrix of each sentence, seriously affecting the memory 
and computing resources. Mainstream methods can map 
data into vector space and operate on vectors to complete 
specific  tasks[6-9][13].  However,  the  existing  vector 
space  model-based  document  processing  methods  are 
based  on  word  frequency  information[10][11].  The 
similarity  of  two  documents  depends  on  the  number  of 
common  words,  and  the  semantic  ambiguity  of  natural 
language cannot be distinguished. 
Deep learning-based representation learning has received 
extensive attention recently[12][14-16]. The fundamental 
drawback is that the neural language model only focuses 
on  the  text  semantics  information  of  academic 
documents[17]  while  ignoring  the  relationship  between 
academic  documents.  Some  researchers  solve  the 
problem  of  imperfect  data  characteristics  by  means  of 
multi-agent  [18-19].  More  and  more  researchers  are 
fusing different features to better complete deep learning 
tasks[20-24]. 
In view of this, graph neural network [25] is proposed to 
extract  the  relationship  structure  information  between 
documents from the document citation network and fuse 
them with the semantic information of the document text, 
so as to construct the representation vector of academic 
documents.  However,  most  of  the  existing  studies  use 
supervised  graph  neural  networks  to  learn  feature 
representations  of  documents[26][27],  which  have 
following two drawbacks: For specific tasks, supervised 
graph neural networks need to develop a huge amount of 
excellently  labeled  data;  The  feature  representation  of 
text  obtained  by  supervised  graph  neural  network[28]is 
highly coupled with the task of labeling datasets, and it is 
difficult  to  directly  transfer  to  other  tasks,  resulting  in 
poor universality of feature representation. 
Compared  to  supervised  learning  method,  unsupervised 
graph neural networks perform better. Because they can 
directly  learn  general  document  feature  representations 
from unlabeled document network data[29].   
Based  on  this,  this  paper  proposes  an  unsupervised 
semantic  representation  learning  method  for  scientific 
literature  based  on  graph  attention  mechanism  and 
maximum mutual information (GAMMI). The following 
are this paper’s main contributions: 
1)  A  semantic  representation  learning  method  for 
scientific literature based on graph attention mechanism 
and  maximum  mutual  information  is  proposed.  By 

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

*Correspondingauthor:YawenLi(warmly0716@126.com).UnsupervisedSemanticRepresentationLearningofScientificLiteratureBasedonGraphAttentionMechanismandMaximumMutualInformationHongruiGao1,YawenLi2*,MeiyuLiang1,ZeliGuan11BeijingKeyLaboratoryofIntelligentCommunicationSoftwareandMultimedia,SchoolofComputerSc...

展开>> 收起<<

Corresponding author Yawen Li warmly0716126.com. Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Corresponding author Yawen Li warmly0716126.com. Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: