Hongwei Wang
Jure Leskovec
ABSTRACT
Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. In this work, we propose a relational message passing method for knowledge graph completion. Different from existing embedding-based methods, relational message passing only considers edge features (i.e., relation types) without entity IDs in the knowledge graph, and passes relational messages among edges iteratively to aggregate neighborhood information. Specifically, two kinds of neighborhood topology are modeled for a given entity pair under the relational message passing framework: (1) Relational context, which captures the relation types of edges adjacent to the given entity pair; (2) Relational paths, which characterize the relative position between the given two entities in the knowledge graph. The two message passing modules are combined together for relation prediction. Experimental results on knowledge graph benchmarks as well as our newly proposed dataset show that, our method PathCon outperforms state-of-the-art knowledge graph completion methods by a large margin. PathCon is also shown applicable to inductive settings where entities are not seen in training stage, and it is able to provide interpretable explanations for the predicted results. The code and all datasets are available at https://github.com/hwwang55/PathCon.
- Monica Bianchini, Marco Gori, and Franco Scarselli. 2001. Processing directed acyclic graphs with recursive neural networks. IEEE Transactions on Neural Networks, Vol. 12, 6 (2001), 1464--1470.Google Scholar
Digital Library
- Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD. 1247--1250.Google Scholar
- Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NeurIPS. 2787--2795.Google Scholar
- Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In AAAI. 301--306.Google Scholar
- Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI. 1306--1313.Google Scholar
- Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In AAAI. 1811--1818.Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint (2018).Google Scholar
- David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints. In NeurIPS. 2224--2232.Google Scholar
- Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M Suchanek. 2015. Fast rule mining in ontological knowledge bases with amie. The VLDB Journal, Vol. 24, 6 (2015), 707--730.Google ScholarDigital Library
- Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML. 1263--1272.Google Scholar
- Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1024--1034.Google Scholar
- Vinh Thinh Ho, Daria Stepanova, Mohamed H Gad-Elrab, Evgeny Kharlamov, and Gerhard Weikum. 2018. Rule learning from knowledge graphs guided by embedding models. In International Semantic Web Conference. Springer, 72--90.Google ScholarDigital Library
- Seyed Mehran Kazemi and David Poole. 2018. Simple embedding for link prediction in knowledge graphs. In NeurIPS. 4284--4295.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2015. Adam: a method for stochastic optimization. In ICLR.Google Scholar
- Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.Google Scholar
- Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. 2020. Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning. In NeurIPS.Google Scholar
- Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2016. Gated graph sequence neural networks. In ICLR.Google Scholar
- George A Miller. 1995. Wordnet: a lexical database for English. Commun. ACM, Vol. 38, 11 (1995), 39--41.Google ScholarDigital Library
- Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, and Daisy Zhe Wang. 2019. Drum: end-to-end differentiable rule mining on knowledge graphs. In NeurIPS. 15321--15331.Google Scholar
- Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference. Springer, 593--607.Google ScholarDigital Library
- Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NeurIPS. 926--934.Google Scholar
- Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: knowledge graph embedding by relational rotation in complex space. In ICLR.Google Scholar
- Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Workshop on Continuous Vector Space Models and their Compositionality. 57--66.Google ScholarCross Ref
- Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In ICML.Google Scholar
- Hongwei Wang and Jure Leskovec. 2020. Unifying graph convolutional neural networks and label propagation. arXiv preprint (2020).Google Scholar
- Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, and Qi Liu. 2018a. Shine: Signed heterogeneous information network embedding for sentiment link prediction. In WSDM. 592--600.Google Scholar
- Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018b. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In CIKM. 417--426.Google ScholarDigital Library
- Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2019 a. Exploring high-order user preference on the knowledge graph for recommender systems. ACM Transactions on Information Systems, Vol. 37, 3 (2019), 1--26.Google ScholarDigital Library
- Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018c. DKN: Deep knowledge-aware network for news recommendation. In WWW. 1835--1844.Google ScholarDigital Library
- Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. 2019 b. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In KDD. 968--977.Google Scholar
- Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2019 c. Multi-task feature learning for knowledge graph enhanced recommendation. In WWW. 2000--2010.Google Scholar
- Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019 d. Knowledge graph convolutional networks for recommender systems. In WWW.Google Scholar
- Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. Deeppath: a reinforcement learning method for knowledge graph reasoning. In EMNLP. 564--573.Google Scholar
- Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks?. In ICLR.Google Scholar
- Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In ICLR.Google Scholar
- Fan Yang, Zhilin Yang, and William W Cohen. 2017. Differentiable learning of logical rules for knowledge base reasoning. In NeurIPS. 2319--2328.Google Scholar
- Jonathan S Yedidia, William T Freeman, and Yair Weiss. 2003. Understanding belief propagation and its generalizations. Exploring artificial intelligence in the new millennium, Vol. 8 (2003), 236--239.Google ScholarDigital Library
- Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In NeurIPS. 5165--5175.Google Scholar
- Shuai Zhang, Yi Tay, Lina Yao, and Qi Liu. 2019 b. Quaternion knowledge graph embeddings. In NeurIPS. 2731--2741.Google Scholar
- Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, and Huajun Chen. 2019 a. Iteratively learning embeddings and rules for knowledge graph reasoning. In WWW. 2366--2377.Google Scholar
Index Terms
- Relational Message Passing for Knowledge Graph Completion
Recommendations
Learning to Walk across Time for Interpretable Temporal Knowledge Graph Completion
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningStatic knowledge graphs (KGs), despite their wide usage in relational reasoning and downstream tasks, fall short of realistic modeling of knowledge and facts that are only temporarily valid. Compared to static knowledge graphs, temporal knowledge graphs ...
Multi-relational knowledge graph completion method with local information fusion
AbstractKnowledge graph completion(KGC) has attracted increasing attention in recent years, aiming at complementing missing relationships between entities in a Knowledge Graph(KG). While the existing KGC approaches utilizing the knowledge within KG could ...
Two-View Graph Neural Networks for Knowledge Graph Completion
The Semantic WebAbstractWe present an effective graph neural network (GNN)-based knowledge graph embedding model, which we name WGE, to capture entity- and relation-focused graph structures. Given a knowledge graph, WGE builds a single undirected entity-focused graph ...
Comments