Denoising Heterogeneous Graph Pre-training Framework for Recommendation

Lei Sang; Yiwen Zhang; Xindong Wu

doi:10.1145/3706632

journal article Jul 10, 2025

Denoising Heterogeneous Graph Pre-training Framework for Recommendation

Lei Sang

Yiwen Zhang

Xindong Wu

ACM Transactions on Information Systems Vol. 43 No. 5 pp. 1-31 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3706632

Abstract

Heterogeneous graph neural networks (HGNN)
have exhibited significant performance gains by modeling the information propagation process in graph-structured data for recommender systems. However, existing HGNN-based Recommendation still face two challenges: (1) They overlook the rich semantics brought by the combination of different meta-paths, making it difficult to capture the importance of various meta-paths; (2) when HGNN use meta-paths to capture high-order information, they are susceptible to noise data, as noise from connected nodes can create cumulative effects on a target node in the graph. To tackle these issues, we propose a new model called the
Denoising Heterogeneous Graph Pre-training Framework (DHGPF)
to enhance recommendation tasks. This framework has two stages: pre-training and training. In the pre-training stage, we assign learnable weights to different meta-paths and use a simplified multi-layer graph convolution network to automatically aggregate semantic information from different meta-path combinations. This approach can capture the importance of these paths. The training stage focuses on reducing noise using gating mechanism and denoising structure learning methods. These methods accomplish the denoising process through information filtering. Our model was evaluated on three real-world datasets, demonstrating that DHGPF outperforms other state-of-the-art recommendation methods. We have further organized the source code of the article at
https://github.com/wangyu0627/DHGPF
.

Topics

No keywords indexed for this article. Browse by subject →

References

69

[1]

Thomas Bird Julius Kunze and David Barber. 2018. Stochastic variational optimization. arXiv:1809.04855. Retrieved from https://arxiv.org/abs/1809.04855

[2]

10.1109/tmm.2022.3151026

[3]

10.1609/aaai.v35i5.16515

[4]

Huiyuan Chen Lan Wang Yusan Lin Chin-Chia Michael Yeh Fei Wang and Hao Yang. 2021b. Structured graph convolutional networks with stochastic masks for recommender systems. In SIGIR 614–623. 10.1145/3404835.3462868

[5]

Lei Chen, Le Wu, Kun Zhang, Richang Hong, Defu Lian, Zhiqiang Zhang, Jun Zhou, and Meng Wang. 2023. Improving recommendation fairness via data augmentation. In WWW, 1012–1020.

[6]

Mengru Chen, Chao Huang, Lianghao Xia, Wei Wei, Yong Xu, and Ronghua Luo. 2023. Heterogeneous graph contrastive learning for recommendation. In WSDM, 544–552.

[7]

Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https://arxiv.org/abs/1810.04805

[8]

10.1007/s00521-022-07251-z

[9]

Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. Metapath2vec: Scalable representation learning for heterogeneous networks. In KDD, 135–144.

[10]

Yuxiao Dong, Ziniu Hu, Kuansan Wang, Yizhou Sun, and Jie Tang. 2020. Heterogeneous network representation learning. In IJCAI, Vol. 20, 4861–4867.

[11]

Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided heterogeneous graph neural network for intent recommendation. In KDD, 2478–2486.

[12]

Hui Han, Tianyu Zhao, Cheng Yang, Hongyi Zhang, Yaoqi Liu, Xiao Wang, and Chuan Shi. 2022. OpenHGNN: An open source toolkit for heterogeneous graph neural network. In CIKM, 3993–3997.

[13]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR, 639–648.

[14]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW, 173–182.

[15]

10.1109/icdm.2008.22

[16]

Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. 2020b. GPT-GNN: Generative pre-training of graph neural networks. In KDD, 1857–1867.

[17]

10.1145/3366423.3380027

[18]

Xunqiang Jiang, Tianrui Jia, Yuan Fang, Chuan Shi, Zhe Lin, and Hui Wang. 2021. Pre-training on large-scale heterogeneous graph. In KDD, 756–766.

[19]

Xunqiang Jiang, Yuanfu Lu, Yuan Fang, and Chuan Shi. 2021. Contrastive pre-training of GNNs on heterogeneous graphs. In CIKM, 803–812.

[20]

Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. 2020. Graph structure learning for robust graph neural networks. In KDD, 66–74.

[21]

Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv:1609.02907. Retrieved from https://arxiv.org/abs/1609.02907

[22]

Zihan Lin, Changxin Tian, Yupeng Hou, and Wayne Xin Zhao. 2022. Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In WWW, 2320–2329.

[23]

Xiaoling Long, Chao Huang, Yong Xu, Huance Xu, Peng Dai, Lianghao Xia, and Liefeng Bo. 2021. Social recommendation with Self-supervised metagraph informax network. In CIKM, 1160–1169.

[24]

Christos Louizos Max Welling and Diederik P Kingma. 2017. Learning sparse neural networks through \(l\_0\) regularization. arXiv:1712.01312. Retrieved from https://arxiv.org/abs/1712.01312

[25]

10.1145/3437963.3441734

[26]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In KDD, 701–710.

[27]

Yuxiang Ren, Bo Liu, Chao Huang, Peng Dai, Liefeng Bo, and Jiawei Zhang. 2019. Heterogeneous deep graph infomax. In AAAI.

[28]

Steffen Rendle Christoph Freudenthaler Zeno Gantner and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv:1205.2618. Retrieved from https://arxiv.org/abs/1205.2618

[29]

Yu Rong Wenbing Huang Tingyang Xu and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification. arXiv:1907.10903. Retrieved from https://arxiv.org/abs/1907.10903

[30]

ImageNet Large Scale Visual Recognition Challenge

Olga Russakovsky, Jia Deng, Hao Su et al.

International Journal of Computer Vision 10.1007/s11263-015-0816-y

[31]

Lei Sang Yu Wang Yi Zhang Yiwen Zhang and Xindong Wu. 2024. Intent-guided heterogeneous graph contrastive learning for recommendation. arXiv:2407.17234. Retrieved from https://arxiv.org/abs/2407.17234

[32]

Context-Dependent Propagating-Based Video Recommendation in Multimodal Heterogeneous Information Networks

Lei Sang, Min Xu, Shengsheng Qian et al.

IEEE Transactions on Multimedia 10.1109/tmm.2020.3007330

[33]

10.1145/371920.372071

[34]

10.1007/978-3-319-93417-4_38

[35]

10.1109/tkde.2018.2833443

[36]

10.1109/tnnls.2023.3273255

[37]

10.1145/3618107

[38]

Lichao Sun, Yingtong Dou, Carl Yang, Kai Zhang, Ji Wang, S. Yu Philip, Lifang He, and Bo Li. 2022. Adversarial attack and defense on graph data: A survey. IEEE Transactions on Knowledge and Data Engineering 35, 8 (2022), 7693–7711.

[39]

Michael Tschannen Josip Djolonga Paul K. Rubenstein Sylvain Gelly and Mario Lucic. 2019. On mutual information maximization for representation learning. arXiv:1907.13625. Retrieved from https://arxiv.org/abs/1907.13625

[40]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008).

[41]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS, 6000–6010.

[42]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. STAT 1050, 20 (2017), 10–48550.

[43]

Petar Veličković William Fedus William L. Hamilton Pietro Liò Yoshua Bengio and R. Devon Hjelm. 2018. Deep graph infomax. arXiv:1809.10341. Retrieved from https://arxiv.org/abs/1809.10341

[44]

10.1145/3528667

[45]

Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In ICML. PMLR, 9929–9939.

[46]

Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, and Tat-Seng Chua. 2021. Deconfounded recommendation for alleviating Bias amplification. In KDD, 1717–1725.

[47]

Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, Min Lin, and Tat-Seng Chua. 2022. Causal representation learning for out-of-distribution recommendation. In WWW, 3562–3571.

[48]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge graph attention network for recommendation. In KDD, 950–958.

[49]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR, 165–174.

[50]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous graph attention network. In WWW, 2022–2032.

Showing 50 of 69 references

Metrics

12

Citations

69

References

Details

Published: Jul 10, 2025
Vol/Issue: 43(5)
Pages: 1-31

Authors

L

Lei Sang

Anhui University, Hefei, China

Y

Yiwen Zhang

Anhui University, Hefei, China

X

Xindong Wu

Hefei University of Technology, Hefei, China

Funding

National Natural Science Foundation of China Award: 62206002, 62206004,and 62272001

Hefei Key Common Technology Project Award: 2023SGJ014

Xunfei Zhiyuan Digital Transformation Innovation Research Special for Universities Award: 2023ZY001

Cite This Article

Lei Sang, Yiwen Zhang, Xindong Wu (2025). Denoising Heterogeneous Graph Pre-training Framework for Recommendation. ACM Transactions on Information Systems, 43(5), 1-31. https://doi.org/10.1145/3706632

Denoising Heterogeneous Graph Pre-training Framework for Recommendation

You May Also Like