ConstGCN Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction Ji Qi1 Bin Xu1 Kaisheng Zeng1 Jinxin Liu1

2025-04-27 0 0 2.48MB 12 页 10玖币

侵权投诉

ConstGCN: Constrained Transmission-based Graph Convolutional

Networks for Document-level Relation Extraction

Ji Qi1, Bin Xu1, Kaisheng Zeng1, Jinxin Liu1,

Jifan Yu1,Qi Gao2,Juanzi Li1,Lei Hou1

1Department of Computer Science and Technology, Tsinghua University

2Beijing Kedong Electric Control System Co. Ltd.

qj20@mails.tsinghua.edu.cn,xubin@tsinghua.edu.cn

Abstract

Document-level relation extraction with graph

neural networks faces a fundamental graph

construction gap between training and infer-

ence - the golden graph structure only avail-

able during training, which causes that most

methods adopt heuristic or syntactic rules to

construct a prior graph as a pseudo proxy. In

this paper, we propose ConstGCN, a novel

graph convolutional network which performs

knowledge-based information propagation be-

tween entities along with all speciﬁc rela-

tion spaces without any prior graph construc-

tion. Speciﬁcally, it updates the entity rep-

resentation by aggregating information from

all other entities along with each relation

space, thus modeling the relation-aware spa-

tial information. To control the information

ﬂow passing through the indeterminate rela-

tion spaces, we propose to constrain the prop-

agation using transmitting scores learned from

the Noise Contrastive Estimation between

fact triples. Experimental results show that

our method outperforms the previous state-

of-the-art (SOTA) approaches on the DocRE

dataset. The source code is publicly available

at https://github.com/THU-KEG/ConstGCN.

1 Introduction

Document-level relation extraction (DocRE) aims

to extract heterogeneous relational graphs of form

{G = (E,R)}

in document, where the typed en-

tities as nodes and multiple directional semantic

relations as edges. In contrast to sentence-level

RE (Qin et al.,2018;Gao et al.,2020), DocRE

has been a growing interest by extracting relations

beyond the sentence boundaries(Yao et al.,2019)

to ensure the information integrity.

Previous DocRE methods tend to apply the graph

neural networks (GNNs) (Kipf and Welling,2016;

Veliˇ

ckovi´

c et al.,2017) as the core component, and

numerous variant of GNNs have proposed (Guo

et al.,2019;Christopoulou et al.,2019;Zeng et al.,

William H. Nobles (1816 – December 28, 1876) was an American

military ofﬁcer, businessman, and politician. Nobles was born in

Genesee County . … In 1848, Nobles settled in Saint Paul ,

Wisconsin Territory . … He also served on the Saint Paul City

Council in 1855 and 1856. … Nobles died in Saint Paul , Minnesota.

country of citizenship

place of death





applies to jurisdiction



contains administrative territorial entity

31 4

614

<latexit sha1_base64="3g3cX6GiENbJzL6aysxDBYWcQc4=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6cdmCfYAtkkyn7dC8SCZKKRW3/oBb/SvxD/QvvDOmoBbRCUnOnHvPmbn3upEnEmlZrzljbn5hcSm/XFhZXVvfKG5uNZIwjRmvs9AL45brJNwTAa9LIT3eimLu+K7Hm+7wXMWbNzxORBhcylHEO77TD0RPMEcS1W77jhy4vTGfXB9dF0tW2dLLnAV2BkrIVjUsvqCNLkIwpPDBEUAS9uAgoecKNixExHUwJi4mJHScY4ICaVPK4pThEDukb592Vxkb0F55JlrN6BSP3piUJvZIE1JeTFidZup4qp0V+5v3WHuqu43o72ZePrESA2L/0k0z/6tTtUj0cKprEFRTpBlVHctcUt0VdXPzS1WSHCLiFO5SPCbMtHLaZ1NrEl276q2j4286U7Fqz7LcFO/qljRg++c4Z0HjoGwfl+3aYalylo06jx3sYp/meYIKLlBFnbwjPOIJz0bNuDXujPvPVCOXabbxbRkPH1DrlDg=</latexit>

<latexit sha1_base64="hKK2+ehg6tAiIQL7L+DRLZVOaqc=">AAACz3icjVHLSsNAFD3GV62vqks3wSK4KomIuiy6cdmCfUBbSpJO26F5MZkopVTc+gNu9a/EP9C/8M6YglpEJyQ5c+49Z+be68Y+T6RlvS4Yi0vLK6u5tfz6xubWdmFnt55EqfBYzYv8SDRdJ2E+D1lNcumzZiyYE7g+a7ijSxVv3DCR8Ci8luOYdQJnEPI+9xxJVLsdOHLo9ids2rW7haJVsvQy54GdgSKyVYkKL2ijhwgeUgRgCCEJ+3CQ0NOCDQsxcR1MiBOEuI4zTJEnbUpZjDIcYkf0HdCulbEh7ZVnotUeneLTK0hp4pA0EeUJwuo0U8dT7azY37wn2lPdbUx/N/MKiJUYEvuXbpb5X52qRaKPc10Dp5pizajqvMwl1V1RNze/VCXJISZO4R7FBWFPK2d9NrUm0bWr3jo6/qYzFav2Xpab4l3dkgZs/xznPKgfl+zTkl09KZYvslHnsI8DHNE8z1DGFSqokXeMRzzh2agat8adcf+Zaixkmj18W8bDB0drlDQ=</latexit>

<latexit sha1_base64="mf74j2/IW0Se52Xig7GUXusDehM=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl047IF+4BWSpJO26F5MZkopVTc+gNu9a/EP9C/8M6YglpEJyQ5c+49Z+be68Y+T6RlveaMhcWl5ZX8amFtfWNzq7i900iiVHis7kV+JFqukzCfh6wuufRZKxbMCVyfNd3RhYo3b5hIeBReyXHMrgNnEPI+9xxJVKcTOHLo9ids2j3uFktW2dLLnAd2BkrIVjUqvqCDHiJ4SBGAIYQk7MNBQk8bNizExF1jQpwgxHWcYYoCaVPKYpThEDui74B27YwNaa88E6326BSfXkFKEwekiShPEFanmTqeamfF/uY90Z7qbmP6u5lXQKzEkNi/dLPM/+pULRJ9nOkaONUUa0ZV52Uuqe6Kurn5pSpJDjFxCvcoLgh7Wjnrs6k1ia5d9dbR8TedqVi197LcFO/qljRg++c450HjqGyflO3acalyno06jz3s45DmeYoKLlFFnbxjPOIJz0bNuDXujPvPVCOXaXbxbRkPH06LlDc=</latexit>

ConstGCNTransE

Vanilla GCN

<latexit sha1_base64="WNZup3jNKI3mkjofzc2joRDPSEo=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIp6rLopsuK9gG1lGQ6rUPTJEwmSimCP+BWP038A/0L74wpqEV0QpIz595zZu69fhyIRDnOa85aWFxaXsmvFtbWNza3its7zSRKJeMNFgWRbPtewgMR8oYSKuDtWHJv7Ae85Y/Odbx1y2UiovBKTWLeHXvDUAwE8xRRl7JX6RVLTtkxy54HbgZKyFY9Kr7gGn1EYEgxBkcIRTiAh4SeDlw4iInrYkqcJCRMnOMeBdKmlMUpwyN2RN8h7ToZG9JeeyZGzeiUgF5JShsHpIkoTxLWp9kmnhpnzf7mPTWe+m4T+vuZ15hYhRti/9LNMv+r07UoDHBqahBUU2wYXR3LXFLTFX1z+0tVihxi4jTuU1wSZkY567NtNImpXffWM/E3k6lZvWdZbop3fUsasPtznPOgeVR2j8vuRaVUPctGncce9nFI8zxBFTXU0SDvIR7xhGerZoVWat19plq5TLOLb8t6+AAI15Ak</latexit>

<latexit sha1_base64="8Id/PkXGBSq/LZXd0VlV5E3rxbY=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl002VF+4BaSjKd1sG8mEyUUgR/wK1+mvgH+hfeGVNQi+iEJGfOvefM3Hv9JBCpcpzXgjU3v7C4VFwurayurW+UN7daaZxJxpssDmLZ8b2UByLiTSVUwDuJ5F7oB7zt35zpePuWy1TE0aUaJ7wXeqNIDAXzFFEXsu/2yxWn6phlzwI3BxXkqxGXX3CFAWIwZAjBEUERDuAhpacLFw4S4nqYECcJCRPnuEeJtBllccrwiL2h74h23ZyNaK89U6NmdEpArySljT3SxJQnCevTbBPPjLNmf/OeGE99tzH9/dwrJFbhmti/dNPM/+p0LQpDnJgaBNWUGEZXx3KXzHRF39z+UpUih4Q4jQcUl4SZUU77bBtNamrXvfVM/M1kalbvWZ6b4V3fkgbs/hznLGgdVN2jqnt+WKmd5qMuYge72Kd5HqOGOhpokvcIj3jCs1W3Iiuz7j5TrUKu2ca3ZT18AAG3kCE=</latexit>

<latexit sha1_base64="tz2z0KwiGBi+J5SN//EPzof1uvE=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRIVdVl002VF+4BaSjKd1tA0CZOJUorgD7jVTxP/QP/CO+MU1CI6IcmZc+85M/dePwmDVDrOa86am19YXMovF1ZW19Y3iptbjTTOBON1FoexaPleysMg4nUZyJC3EsG9kR/ypj88V/HmLRdpEEdXcpzwzsgbREE/YJ4k6lJ0D7vFklN29LJngWtACWbV4uILrtFDDIYMI3BEkIRDeEjpacOFg4S4DibECUKBjnPco0DajLI4ZXjEDuk7oF3bsBHtlWeq1YxOCekVpLSxR5qY8gRhdZqt45l2Vuxv3hPtqe42pr9vvEbEStwQ+5dumvlfnapFoo9TXUNANSWaUdUx45Lprqib21+qkuSQEKdwj+KCMNPKaZ9trUl17aq3no6/6UzFqj0zuRne1S1pwO7Pcc6CxkHZPS67F0elypkZdR472MU+zfMEFVRRQ528B3jEE56tqhVZmXX3mWrljGYb35b18AEGd5Aj</latexit>

Golden Graph

<latexit sha1_base64="HgDj9YZNIBSZuISo1y+q84EI7KE=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdVl047IF+wBbJJlO29C8mEyUUipu/QG3+lfiH+hfeGdMQS2iE5KcOfeeM3PvdWPfS6RlveaMufmFxaX8cmFldW19o7i51UiiVDBeZ5EfiZbrJNz3Ql6XnvR5KxbcCVyfN93huYo3b7hIvCi8lKOYdwKnH3o9jzmSqHY7cOTA7Y3F5PrwuliyypZe5iywM1BCtqpR8QVtdBGBIUUAjhCSsA8HCT1XsGEhJq6DMXGCkKfjHBMUSJtSFqcMh9ghffu0u8rYkPbKM9FqRqf49ApSmtgjTUR5grA6zdTxVDsr9jfvsfZUdxvR3828AmIlBsT+pZtm/lenapHo4VTX4FFNsWZUdSxzSXVX1M3NL1VJcoiJU7hLcUGYaeW0z6bWJLp21VtHx990pmLVnmW5Kd7VLWnA9s9xzoLGQdk+Ltu1o1LlLBt1HjvYxT7N8wQVXKCKOnnHeMQTno2acWvcGfefqUYu02zj2zIePgBrMpRD</latexit>

<latexit sha1_base64="3BS0vNbRhY+kBDmoJNnphBOyPFk=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl047IF+4BWSjKdtqF5kUyUUipu/QG3+lfiH+hfeGecglpEJyQ5c+49Z+be68a+lwrLes0ZC4tLyyv51cLa+sbmVnF7p5FGWcJ4nUV+lLRcJ+W+F/K68ITPW3HCncD1edMdXch484YnqReFV2Ic8+vAGYRe32OOIKrTCRwxdPsTPu0edYslq2ypZc4DW4MS9KpGxRd00EMEhgwBOEIIwj4cpPS0YcNCTNw1JsQlhDwV55iiQNqMsjhlOMSO6DugXVuzIe2lZ6rUjE7x6U1IaeKANBHlJYTlaaaKZ8pZsr95T5SnvNuY/q72CogVGBL7l26W+V+drEWgjzNVg0c1xYqR1THtkqmuyJubX6oS5BATJ3GP4glhppSzPptKk6raZW8dFX9TmZKVe6ZzM7zLW9KA7Z/jnAeNo7J9UrZrx6XKuR51HnvYxyHN8xQVXKKKOnnHeMQTno2acWvcGfefqUZOa3bxbRkPH0nLlDU=</latexit>

<latexit sha1_base64="FxdbzfKNUjk5MxIl/W14uaux3Dg=">AAACz3icjVHLSsNAFD3GV62vqks3wSK4KolIdVl047IF+4BWyiSdtkPzIpkopVTc+gNu9a/EP9C/8M6YglpEJyQ5c+49Z+be60SeSKRlvS4Yi0vLK6u5tfz6xubWdmFnt5GEaezyuht6YdxyWMI9EfC6FNLjrSjmzHc83nRGFyrevOFxIsLgSo4jfu2zQSD6wmWSqE7HZ3Lo9Cd82i13C0WrZOllzgM7A0VkqxoWXtBBDyFcpPDBEUAS9sCQ0NOGDQsRcdeYEBcTEjrOMUWetCllccpgxI7oO6BdO2MD2ivPRKtdOsWjNyaliUPShJQXE1anmTqeamfF/uY90Z7qbmP6O5mXT6zEkNi/dLPM/+pULRJ9nOkaBNUUaUZV52Yuqe6Kurn5pSpJDhFxCvcoHhN2tXLWZ1NrEl276i3T8TedqVi1d7PcFO/qljRg++c450HjuGSXS3btpFg5z0adwz4OcETzPEUFl6iiTt4RHvGEZ6Nm3Bp3xv1nqrGQafbwbRkPH1NLlDk=</latexit>

<latexit sha1_base64="UlOOTzREx9USg+FzvORCScG+re0=">AAACz3icjVHLSsNAFD3GV62vqks3wSK4KomIuiy6cdmCfUBbSpJO26F5MZkopVTc+gNu9a/EP9C/8M6YglpEJyQ5c+49Z+be68Y+T6RlvS4Yi0vLK6u5tfz6xubWdmFnt55EqfBYzYv8SDRdJ2E+D1lNcumzZiyYE7g+a7ijSxVv3DCR8Ci8luOYdQJnEPI+9xxJVLsdOHLo9idi2rW7haJVsvQy54GdgSKyVYkKL2ijhwgeUgRgCCEJ+3CQ0NOCDQsxcR1MiBOEuI4zTJEnbUpZjDIcYkf0HdCulbEh7ZVnotUeneLTK0hp4pA0EeUJwuo0U8dT7azY37wn2lPdbUx/N/MKiJUYEvuXbpb5X52qRaKPc10Dp5pizajqvMwl1V1RNze/VCXJISZO4R7FBWFPK2d9NrUm0bWr3jo6/qYzFav2Xpab4l3dkgZs/xznPKgfl+zTkl09KZYvslHnsI8DHNE8z1DGFSqokXeMRzzh2agat8adcf+Zaixkmj18W8bDB2ZylEE=</latexit>

<latexit sha1_base64="j3LVv5BwqPj5agU4FmWnK8e8ch0=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl047IF+4BWSpJO26F5MZkopVTc+gNu9a/EP9C/8M6YglpEJyQ5c+49Z+be68Y+T6RlveaMhcWl5ZX8amFtfWNzq7i900iiVHis7kV+JFqukzCfh6wuufRZKxbMCVyfNd3RhYo3b5hIeBReyXHMrgNnEPI+9xxJVKcTOHLo9idi2j3uFktW2dLLnAd2BkrIVjUqvqCDHiJ4SBGAIYQk7MNBQk8bNizExF1jQpwgxHWcYYoCaVPKYpThEDui74B27YwNaa88E6326BSfXkFKEwekiShPEFanmTqeamfF/uY90Z7qbmP6u5lXQKzEkNi/dLPM/+pULRJ9nOkaONUUa0ZV52Uuqe6Kurn5pSpJDjFxCvcoLgh7Wjnrs6k1ia5d9dbR8TedqVi197LcFO/qljRg++c450HjqGyflO3acalyno06jz3s45DmeYoKLlFFnbxjPOIJz0bNuDXujPvPVCOXaXbxbRkPH22SlEQ=</latexit>

<latexit sha1_base64="epW065DkCijuNdlx1J2jzq8XnGo=">AAACzHicjVHLSsNAFD2Nr1pfVZdugkUQhJJIUZdFN66kgn1IrSWZTmtoXiQToZRu/QG3+l3iH+hfeGecglpEJyQ5c+49Z+be68a+lwrLes0Zc/MLi0v55cLK6tr6RnFzq5FGWcJ4nUV+lLRcJ+W+F/K68ITPW3HCncD1edMdnsl4854nqReFV2IU807gDEKv7zFHEHXNb8fiwJ50K91iySpbapmzwNagBL1qUfEFN+ghAkOGABwhBGEfDlJ62rBhISaugzFxCSFPxTkmKJA2oyxOGQ6xQ/oOaNfWbEh76ZkqNaNTfHoTUprYI01EeQlheZqp4plyluxv3mPlKe82or+rvQJiBe6I/Us3zfyvTtYi0MeJqsGjmmLFyOqYdslUV+TNzS9VCXKIiZO4R/GEMFPKaZ9NpUlV7bK3joq/qUzJyj3TuRne5S1pwPbPcc6CxmHZPirbl5VS9VSPOo8d7GKf5nmMKs5RQ528AzziCc/GhSGMsTH5TDVyWrONb8t4+ADzbZJ5</latexit>

et+1

<latexit sha1_base64="h1xuA+6I1ZXKacLcjsHtYrGRP8g=">AAACznicjVHLSsNAFD2Nr1pfVZdugkUQhJJIUZdFNy4r2AfUWpLptIbmxWRSKKW49Qfc6meJf6B/4Z0xBbWITkhy5txz7sy91419L5GW9ZozFhaXllfyq4W19Y3NreL2TiOJUsF4nUV+JFquk3DfC3ldetLnrVhwJ3B93nSHFyreHHGReFF4Lccx7wTOIPT6HnMkUW1+K7u2eWSKbqVbLFllSy9zHtgZKCFbtaj4ghv0EIEhRQCOEJKwDwcJPW3YsBAT18GEOEHI03GOKQrkTUnFSeEQO6TvgHbtjA1pr3Im2s3oFJ9eQU4TB+SJSCcIq9NMHU91ZsX+lnuic6q7jenvZrkCYiXuiP3LN1P+16dqkejjTNfgUU2xZlR1LMuS6q6om5tfqpKUISZO4R7FBWGmnbM+m9qT6NpVbx0df9NKxao9y7Qp3tUtacD2z3HOg8Zx2T4p21eVUvU8G3Uee9jHIc3zFFVcooa67vgjnvBs1IyRMTXuP6VGLvPs4tsyHj4AfpSSpg==</latexit>

1+r4

<latexit sha1_base64="Oq1ozwm6/hv0cNj8f5Wbduvf3nc=">AAACznicjVHLSsNAFD2Nr1pfVZdugkUQhJKIVJdFNy4r2AfUWpLptIbmxWRSKKW49Qfc6meJf6B/4Z0xBbWITkhy5txz7sy91419L5GW9ZozFhaXllfyq4W19Y3NreL2TiOJUsF4nUV+JFquk3DfC3ldetLnrVhwJ3B93nSHFyreHHGReFF4Lccx7wTOIPT6HnMkUW1+K7sV88gUXbtbLFllSy9zHtgZKCFbtaj4ghv0EIEhRQCOEJKwDwcJPW3YsBAT18GEOEHI03GOKQrkTUnFSeEQO6TvgHbtjA1pr3Im2s3oFJ9eQU4TB+SJSCcIq9NMHU91ZsX+lnuic6q7jenvZrkCYiXuiP3LN1P+16dqkejjTNfgUU2xZlR1LMuS6q6om5tfqpKUISZO4R7FBWGmnbM+m9qT6NpVbx0df9NKxao9y7Qp3tUtacD2z3HOg8Zx2a6U7auTUvU8G3Uee9jHIc3zFFVcooa67vgjnvBs1IyRMTXuP6VGLvPs4tsyHj4Ag3KSqA==</latexit>

6+r1

<latexit sha1_base64="1VywvpD9Wp12CM2nHoOmxldvew4=">AAACznicjVHLSsNAFD2Nr1pfVZdugkVwY0lU1GXRjcsK9gG1lmQ6raF5MZkUSilu/QG3+lniH+hfeGdMQS2iE5KcOfecO3PvdWPfS6RlveaMufmFxaX8cmFldW19o7i5VU+iVDBeY5EfiabrJNz3Ql6TnvR5MxbcCVyfN9zBhYo3hlwkXhRey1HM24HTD72exxxJVIvfys6heWCKzlGnWLLKll7mLLAzUEK2qlHxBTfoIgJDigAcISRhHw4SelqwYSEmro0xcYKQp+McExTIm5KKk8IhdkDfPu1aGRvSXuVMtJvRKT69gpwm9sgTkU4QVqeZOp7qzIr9LfdY51R3G9HfzXIFxErcEfuXb6r8r0/VItHDma7Bo5pizajqWJYl1V1RNze/VCUpQ0ycwl2KC8JMO6d9NrUn0bWr3jo6/qaVilV7lmlTvKtb0oDtn+OcBfXDsn1Stq+OS5XzbNR57GAX+zTPU1RwiSpquuOPeMKzUTWGxsS4/5QaucyzjW/LePgAg2KSqA==</latexit>

2r3

et+1

<latexit sha1_base64="6l1Xo+60iKYDKQkWTBqnF6/6F4E=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl0I64qmLagtSTTaR2aJmEyUUpx4w+41S8T/0D/wjtjCmoRnZDkzLn3nJl7b5CEIlWO81qwZmbn5heKi6Wl5ZXVtfL6RiONM8m4x+Iwlq3AT3koIu4poULeSiT3h0HIm8HgRMebt1ymIo4u1Cjh7aHfj0RPMF8R5fFr1XE75YpTdcyyp4GbgwryVY/LL7hCFzEYMgzBEUERDuEjpecSLhwkxLUxJk4SEibOcY8SaTPK4pThEzugb592lzkb0V57pkbN6JSQXklKGzukiSlPEtan2SaeGWfN/uY9Np76biP6B7nXkFiFG2L/0k0y/6vTtSj0cGRqEFRTYhhdHctdMtMVfXP7S1WKHBLiNO5SXBJmRjnps200qald99Y38TeTqVm9Z3luhnd9Sxqw+3Oc06CxV3UPqu75fqV2nI+6iC1sY5fmeYgaTlGHR94Cj3jCs3VmJdadNfpMtQq5ZhPflvXwASPxkPo=</latexit>

<latexit sha1_base64="hbFYdtSVtnoj4OGVTxdZjH7rHBM=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6EVcVTFuotSTTaR2aF8lEKcWNP+BWv0z8A/0L74wpqEV0QpIz595zZu69XuyLVFrWa8GYmZ2bXygulpaWV1bXyusbjTTKEsYdFvlR0vLclPsi5I4U0uetOOFu4Pm86Q1PVbx5y5NUROGlHMW8E7iDUPQFcyVRDr+W3YNuuWJVLb3MaWDnoIJ81aPyC67QQwSGDAE4QkjCPlyk9LRhw0JMXAdj4hJCQsc57lEibUZZnDJcYof0HdCunbMh7ZVnqtWMTvHpTUhpYoc0EeUlhNVppo5n2lmxv3mPtae624j+Xu4VECtxQ+xfuknmf3WqFok+jnUNgmqKNaOqY7lLpruibm5+qUqSQ0ycwj2KJ4SZVk76bGpNqmtXvXV1/E1nKlbtWZ6b4V3dkgZs/xznNGjsVe3Dqn2xX6md5KMuYgvb2KV5HqGGM9ThkLfAI57wbJwbsXFnjD5TjUKu2cS3ZTx8AC1xkP4=</latexit>

Figure 1: An example of DocRED document (bottom)

with one of its golden multi-relational graphs (upper

left). On the upper right, compared to the vanilla GNNs

updating entity by accumulating representations of syn-

tactically adjacent entities on the pseudo graph, the

proposed ConstGNN models the relation-aware struc-

tural representation of entity by performing knowledge-

based information propagation.

2020;Xu et al.,2021). Similar to traditional GNNs

modeled on the observable graph structures (e.g.

social networks (Huang et al.,2019) and academic

citation network (Feng et al.,2020)), these mod-

els all require a pre-speciﬁed graph construction.

They mainly either rely on the heuristic rules of

intra(inter)-sentential information of entities and

mentions (Zeng et al.,2020;Christopoulou et al.,

2019), or leverage the syntactic rules of depen-

dency paths built by an external parser (Sahu et al.,

2019;Guo et al.,2019) to serve as the prior graph

structure of GNNs. We consider such graph struc-

ture as a pseudo graph structure, for it establishes

each edge between a pair of entities as a binary

association based on the task-independent auxiliary

information (heuristic/syntactic rules).

However, the golden edges that describe the re-

lationships between two entities contain multi-type

abundant semantics. Thus, previous approaches

suffer from two major intrinsic issues. First,

Hin-

dered Propagation Issue

: the construction of the

arXiv:2210.03949v1 [cs.CL] 8 Oct 2022

pseudo graph structure ignores many actual rela-

tional edges between entities, which hinders the

effective information acquisition and dissemina-

tion. Second,

Noisy Representation Issue

: the

simple information accumulation based on the bi-

nary associative edges on the pseudo graph makes

them struggle to model relation-aware structural

knowledge, which further results in noisy represen-

tations and harms the performance. For example,

in Figure 1, the entity Saint Paul’s representaion

is updated by simply accumulating the representa-

tions of its syntactically adjacent entities, making it

similar to entities Nobles and Wisconsin Territory,

while they are completely different entities with

relation connections place_of_birth.

Instead of introducing the prior pseudo

graph structures, we present

ConstGCN

, a

novel

Cons

trained

ransmission-based

raph

onvolutional

etwork that performs knowledge-

based information propagation between entities

along with all relation spaces without any prior

graph construction which explicitly models the se-

mantics of various relationships. Speciﬁcally, we

innovatively propose the knowledge-based infor-

mation propagation for DocRE by leveraging the

ﬂexible Knowledge Graph Embedding (KGE) ap-

proaches (Wang et al.,2017) into a general trans-

mitting operation. At each graph convolution step,

it updates the entity representation by aggregating

knowledge-based information broadcasted from

neighbor entities along with all relation spaces.

Thus, entity representations containing the relation-

aware structural semantics are learned effectively

and directly. Due to the agnostic nature of golden

graph structure in documents, it is difﬁcult to rig-

orously follow the relational paths to transmit in-

formation. We propose the transmitting scores to

constrain the information ﬂow through the inde-

terminate relational edges, where the scores are

learned jointly from the Noise Contrastive Estima-

tion (NCE) (Mikolov et al.,2013). It allows the

model to learn the semantic representations while

maintaining the original relation-aware structural

information. As shown in ﬁgure 1, compared to

the vanilla GNNs, our model learns the represen-

tations with an isomorphic structure to the golden

heterogeneous graph in the document.

We conduct extensive experiments on DocRED,

a large-scale human-annotated dataset including

heterogeneous graphs among entities in each docu-

ment. The results show that our model achieves the

SOTA performance compared to previous methods.

In addition to the proposed graph convolutional

network, we further demonstrated the compatibil-

ity of representation learning from documents and

knowledge graphs. The contributions of our work

are summarized as follows:

•

We present a novel graph convolutional net-

work, ConstGCN, that can naturally model the

heterogeneous graph structure including inde-

terminate edges based on knowledge-based

information propagation.

•

We propose the approach of constrained trans-

mission, which allows the model to learn the

entity representations while maintaining the

original relation-aware structural information.

•

We conduct experiments on the DocRE task

and achieve the SOTA performance. This

work also demonstrates the compatibility of

representation learning from documents and

knowledge graphs.

2 Preliminary

2.1 Document-level Relation Extraction

Given a textual document

D={wi}|D|

i=1

consist-

ing of a sequence of words and a set of typed en-

tities

E={ei}|E|

i=1

, where each entity refers to

a set of mentions

e={mi}|e|

i=1

which is a se-

quence of words in the document. The task of

DocRE aims to extract the heterogeneous graphs

{(ei, rk, ej)|ei, ej∈ E, rk∈ R}

, in which a pair

of entities nodes

ei, ej

may have multiple edges

and each edge

refers to a speciﬁc relation type,

and Ris the set of predeﬁned relations.

2.2 Knowledge Graph Embedding

Given a knowledge graph (KG) consisting of a col-

lection of triples

G={(ei, rk, ej)|ei, ej∈ E, rk∈

with the sets of pre-deﬁned types. The task

of KGE aims to learn the vectorial representations

ei,rk,ej

modeling the heterogeneous structural in-

formation based on a scoring function

. Depend-

ing on the scoring function used, typical methods

can be divided into two categories: the translational

distance-based approaches e.g. (TransE(Bordes

et al.) and RotatE(Sun et al.,2019)) and the

semantic matching-based approaches (e.g. Dist-

Mult(Yang et al.,2014) and ComplEx(Trouillon

et al.,2016)). As shown in table 1, the key idea be-

hind both categories of methods is to transmit the

William H. Nobles (1816 – December 28,

1876) was an American military ofﬁcer …

. Nobles was born in Genesee County … .

In 1848, Nobles settled in Saint Paul ,

Wisconsin Territory . … He also served on

the Saint Paul City Council in 1855 … .

PLM Encoder

<latexit sha1_base64="62GNLJ8L9VZZiSRLaUdYTG84BQI=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVZIi6rLopsuK9gG1lGQ6rUPTJEwmSimCP+BWP038A/0L74wpqEV0QpIz595zZu69fhyIRDnOa85aWFxaXsmvFtbWNza3its7zSRKJeMNFgWRbPtewgMR8oYSKuDtWHJv7Ae85Y/Odbx1y2UiovBKTWLeHXvDUAwE8xRRl7JX6RVLTtkxy54HbgZKyFY9Kr7gGn1EYEgxBkcIRTiAh4SeDlw4iInrYkqcJCRMnOMeBdKmlMUpwyN2RN8h7ToZG9JeeyZGzeiUgF5JShsHpIkoTxLWp9kmnhpnzf7mPTWe+m4T+vuZ15hYhRti/9LNMv+r07UoDHBqahBUU2wYXR3LXFLTFX1z+0tVihxi4jTuU1wSZkY567NtNImpXffWM/E3k6lZvWdZbop3fUsasPtznPOgWSm7x2X34qhUPctGncce9nFI8zxBFTXU0SDvIR7xhGerZoVWat19plq5TLOLb8t6+AAEF5Ai</latexit>

<latexit sha1_base64="879FA3qr4V2y50oXSpeYtyGl208=">AAACy3icjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl040aoYB9QS0nSaR2aZkJmItTapT/gVv9L/AP9C++MKahFdEKSM+eec2fuvX4ccqkc5zVnzc0vLC7llwsrq2vrG8XNrboUaRKwWiBCkTR9T7KQR6ymuApZM06YN/RD1vAHZzreuGWJ5CK6UqOYtYdeP+I9HniKqGavM46FCCedYskpO2bZs8DNQAnZqoriC67RhUCAFEMwRFCEQ3iQ9LTgwkFMXBtj4hJC3MQZJiiQNyUVI4VH7IC+fdq1Mjaivc4pjTugU0J6E3La2COPIF1CWJ9mm3hqMmv2t9xjk1PfbUR/P8s1JFbhhti/fFPlf326FoUeTkwNnGqKDaOrC7IsqemKvrn9pSpFGWLiNO5SPCEcGOe0z7bxSFO77q1n4m9GqVm9DzJtind9Sxqw+3Ocs6B+UHaPyu7lYalymo06jx3sYp/meYwKzlFFzczxEU94ti4sad1Z959SK5d5tvFtWQ8fn1iSyA==</latexit>

fpool

country of citizenship

place of death





applies to jurisdiction



contains administrative territorial entity

Expected Output Graph

Input Document

<latexit sha1_base64="c+apyjzc+qyZl5y3XBN5IUQKERM=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy66bKifUAtJZlOa2iahMlEKUXwB9zqp4l/oH/hnXEKahGdkOTMufecmXuvn4RBKh3nNWfNzS8sLuWXCyura+sbxc2tRhpngvE6i8NYtHwv5WEQ8boMZMhbieDeyA950x+eq3jzlos0iKMrOU54Z+QNoqAfME8SdSm6R91iySk7etmzwDWgBLNqcfEF1+ghBkOGETgiSMIhPKT0tOHCQUJcBxPiBKFAxznuUSBtRlmcMjxih/Qd0K5t2Ij2yjPVakanhPQKUtrYI01MeYKwOs3W8Uw7K/Y374n2VHcb0983XiNiJW6I/Us3zfyvTtUi0cepriGgmhLNqOqYccl0V9TN7S9VSXJIiFO4R3FBmGnltM+21qS6dtVbT8ffdKZi1Z6Z3Azv6pY0YPfnOGdB46DsHpfdi8NS5cyMOo8d7GKf5nmCCqqooU7eAzziCc9W1YqszLr7TLVyRrONb8t6+AALN5Al</latexit>

<latexit sha1_base64="rWepnJx2EMui7noL/Eg6mzVUk0U=">AAACyHicjVHLSsNAFD2Nr1pfVZdugkVwVRIRdVl0I64qmLZQS0mm0zqYJmEyUUrpxh9wq18m/oH+hXfGFNQiOiHJmXPvOTP33iAJRaoc57Vgzc0vLC4Vl0srq2vrG+XNrUYaZ5Jxj8VhLFuBn/JQRNxTQoW8lUjuD4OQN4PbMx1v3nGZiji6UqOEd4b+IBJ9wXxFlNfvjuWkW644Vccsexa4OaggX/W4/IJr9BCDIcMQHBEU4RA+UnracOEgIa6DMXGSkDBxjglKpM0oi1OGT+wtfQe0a+dsRHvtmRo1o1NCeiUpbeyRJqY8SVifZpt4Zpw1+5v32Hjqu43oH+ReQ2IVboj9SzfN/K9O16LQx4mpQVBNiWF0dSx3yUxX9M3tL1UpckiI07hHcUmYGeW0z7bRpKZ23VvfxN9Mpmb1nuW5Gd71LWnA7s9xzoLGQdU9qrqXh5XaaT7qInawi32a5zFqOEcdHnkLPOIJz9aFlVj31ugz1Srkmm18W9bDBxsokWI=</latexit>

<latexit sha1_base64="zTsGjeRSoPv6ZNDWSLqiKpgujZo=">AAACz3icjVHLSsNAFD2Nr1pfVZdugkVwVRIVdVl047IF+wBbJJlO26F5kUyUUipu/QG3+lfiH+hfeGdMQS2iE5KcOfeeM3PvdSNPJNKyXnPG3PzC4lJ+ubCyura+UdzcaiRhGjNeZ6EXxi3XSbgnAl6XQnq8FcXc8V2PN93huYo3b3iciDC4lKOId3ynH4ieYI4kqt32HTlwe2M+uT68LpassqWXOQvsDJSQrWpYfEEbXYRgSOGDI4Ak7MFBQs8VbFiIiOtgTFxMSOg4xwQF0qaUxSnDIXZI3z7trjI2oL3yTLSa0SkevTEpTeyRJqS8mLA6zdTxVDsr9jfvsfZUdxvR3828fGIlBsT+pZtm/lenapHo4VTXIKimSDOqOpa5pLor6ubml6okOUTEKdyleEyYaeW0z6bWJLp21VtHx990pmLVnmW5Kd7VLWnA9s9xzoLGQdk+Ltu1o1LlLBt1HjvYxT7N8wQVXKCKOnlHeMQTno2acWvcGfefqUYu02zj2zIePgBMK5Q2</latexit>

<latexit sha1_base64="y1IXLFfurJ3mpNKa26zcVeOnNQg=">AAACxnicjVHLSsNAFD2Nr1pfVZdugkVwVRKR6rLopsuK9gG1lGQ6rUPTJEwmSimCP+BWP038A/0L74wpqEV0QpIz595zZu69fhyIRDnOa85aWFxaXsmvFtbWNza3its7zSRKJeMNFgWRbPtewgMR8oYSKuDtWHJv7Ae85Y/Odbx1y2UiovBKTWLeHXvDUAwE8xRRl7JX6RVLTtkxy54HbgZKyFY9Kr7gGn1EYEgxBkcIRTiAh4SeDlw4iInrYkqcJCRMnOMeBdKmlMUpwyN2RN8h7ToZG9JeeyZGzeiUgF5JShsHpIkoTxLWp9kmnhpnzf7mPTWe+m4T+vuZ15hYhRti/9LNMv+r07UoDHBqahBUU2wYXR3LXFLTFX1z+0tVihxi4jTuU1wSZkY567NtNImpXffWM/E3k6lZvWdZbop3fUsasPtznPOgeVR2K2X34rhUPctGncce9nFI8zxBFTXU0SDvIR7xhGerZoVWat19plq5TLOLb8t6+AANl5Am</latexit>

<latexit sha1_base64="Y1OrL3P4utNMXoNwPhVye+97CNE=">AAACz3icjVHLSsNAFD3GV62vqks3wSK4KolIdVl047IF+4BWSpJO29C8mEyUUipu/QG3+lfiH+hfeGecglpEJyQ5c+49Z+be6yaBnwrLel0wFpeWV1Zza/n1jc2t7cLObiONM+6xuhcHMW+5TsoCP2J14YuAtRLOnNANWNMdXch484bx1I+jKzFO2HXoDCK/73uOIKrTCR0xdPsTPu2Wu4WiVbLUMueBrUERelXjwgs66CGGhwwhGCIIwgEcpPS0YcNCQtw1JsRxQr6KM0yRJ21GWYwyHGJH9B3Qrq3ZiPbSM1Vqj04J6OWkNHFImpjyOGF5mqnimXKW7G/eE+Up7zamv6u9QmIFhsT+pZtl/lcnaxHo40zV4FNNiWJkdZ52yVRX5M3NL1UJckiIk7hHcU7YU8pZn02lSVXtsreOir+pTMnKvadzM7zLW9KA7Z/jnAeN45JdLtm1k2LlXI86h30c4IjmeYoKLlFFnbwTPOIJz0bNuDXujPvPVGNBa/bwbRkPH3JSlEY=</latexit>

PLM Encoding Transmitting Score Modeling Knowledge-based Propagation Attentive Pooling

Figure 2: Overview of ConstGCN. Given the representations of entities, ConstGCN ﬁrst computes the transmitting

scores between entities in all relation spaces and then performs the graph convolution that updates each entity

by transmitting its neighboring information along with projection spaces of all relations under the constraints of

transmitting scores. Finally, the entity representations that model the structural semantics of heterogeneous graphs

are learned and further used to predict the relational classes.

Methods Scoring Function

TransE dr(ei, rk, ej) = γ− ||ei+ej−rk||

RotatE dr(ei, rk, ej) = γ−||hei◦ej−rki||

DistMult

dr(ei, rk, ej) = hei,ej,rki

ComplEx

dr(ei, rk, ej) = Re(hei,ej,rki)

Table 1: The scoring functions for typical KGE models.

The L1-norm is used for all distance based models and

the subscript || · ||1is dropped for brevity.

representation of head entity in relation-speciﬁc

projection spaces to approximate tail entity. We

thus deﬁne a uniﬁed transmitting operation

⊕

for

the existing typical KGE approaches,

e⊕r=









e+r,(TransE)

he,ri,(DistMult)

Re(he,ri),(ComplEx)

(1)

where

h·i

denotes the generalized dot product, and

is the operation that returns the real part of a

complex value. The transmitting operation above

will be used to perform knowledge-based message

passing in document.

3 Methodology

In this section, we introduce the details of the pro-

posed model. The overall framework is illustrated

in ﬁgure 2. Based on the representations of enti-

ties obtained from a PLM encoder, ConstGCN is

composed of

graph convolutional layers and each

layer has two computational steps: the computation

of transmitting scores and the computation of mes-

sage passing under the constraints of transmitting

scores.

3.1 PLM Encoding

Given an input document consisting of a sequence

of words with a set of typed entities, we ﬁrst insert a

special token "*" at the start and end of entity men-

tions based on the entity marker tecnique (Zhang

et al.,2017;Shi and Lin,2019;Baldini Soares et al.,

2019). For the processed document

D={xi}|D|

i=1

we then employ a pre-trained language model (e.g.

BERT (Devlin et al.,2019)) to get contextual se-

quence representations:

H= (x1, ..., x|D|) = PLM(x1, ..., x|D|),(2)

where

xi∈Rdw

refer to the contextualized em-

bedding of

-th token. For those documents that

the sequence length are longer than the maximum

input length of encoder, we compute the represen-

tations of overlapping tokens by averaging their

embeddings from different windows. Then, we uti-

lize the embedding of special token "*" at the start

-th mention to represent the mention

. For

-th entity with its mentions

ei={mu}|ei|

u=1

, we

compute an initial coarse-grained entity representa-

tion by averaging its coreference mentions with the

logsumexp pooling function (Zhang et al.,2019):

ei= log X

mu∈ei

exp(mu),(3)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ConstGCN:ConstrainedTransmission-basedGraphConvolutionalNetworksforDocument-levelRelationExtractionJiQi1,BinXu1,KaishengZeng1,JinxinLiu1,JifanYu1,QiGao2,JuanziLi1,LeiHou11DepartmentofComputerScienceandTechnology,TsinghuaUniversity2BeijingKedongElectricControlSystemCo.Ltd.qj20@mails.tsinghua.edu.cn,x...

展开>> 收起<<

ConstGCN Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction Ji Qi1 Bin Xu1 Kaisheng Zeng1 Jinxin Liu1.pdf

共12页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

ConstGCN Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction Ji Qi1 Bin Xu1 Kaisheng Zeng1 Jinxin Liu1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: