BishtRef
Articles Journals Publishers Authors Subjects Most Cited New Papers Year Stats Open Access
journal article Feb 07, 2023

ReFRS: Resource-efficient Federated Recommender System for Dynamic and Diversified User Preferences

Mubashir Imran ORCID Hongzhi Yin ORCID Quoc Viet Hung Nguyen Alexander Zhou ORCID Kai Zheng ORCID
ACM Transactions on Information Systems Vol. 41 No. 3 pp. 1-30 · Association for Computing Machinery (ACM)
View at Publisher Save 10.1145/3560486
Abstract
Owing to its nature of scalability and privacy by design, federated learning (FL) has received increasing interest in decentralized deep learning. FL has also facilitated recent research on upscaling and privatizing personalized recommendation services, using on-device data to learn recommender models locally. These models are then aggregated globally to obtain a more performant model while maintaining data privacy. Typically, federated recommender systems (FRSs) do not take into account the lack of resources and data availability at the end-devices. In addition, they assume that the interaction data between users and items is i.i.d. and stationary across end-devices (i.e., users), and that all local recommender models can be directly averaged without considering the user’s behavioral diversity. However, in real scenarios, recommendations have to be made on end-devices with sparse interaction data and limited resources. Furthermore, users’ preferences are heterogeneous and they frequently visit new items. This makes their personal preferences highly skewed, and the straightforwardly aggregated model is thus ill-posed for such non-i.i.d. data. In this article, we propose
Resource Efficient Federated Recommender System
(ReFRS) to enable decentralized recommendation with dynamic and diversified user preferences. On the device side, ReFRS consists of a lightweight self-supervised local model built upon the variational autoencoder for learning a user’s temporal preference from a sequence of interacted items. On the server side, ReFRS utilizes a scalable semantic sampler to adaptively perform model aggregation within each identified cluster of similar users. The clustering module operates in an asynchronous and dynamic manner to support efficient global model update and cope with shifting user interests. As a result, ReFRS achieves superior performance in terms of both accuracy and scalability, as demonstrated by comparative experiments on real datasets.
Topics

No keywords indexed for this article. Browse by subject →

References
103
[1]
Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions

G. Adomavicius, A. Tuzhilin

IEEE Transactions on Knowledge and Data Engineerin... 10.1109/tkde.2005.99
[3]
Waqar Ali, Rajesh Kumar, Zhiyi Deng, Yansong Wang, and Jie Shao. 2021. A federated learning approach for privacy protection in context-aware recommender systems. Comput. J. 64, 7 (2021), 1016–1027.
[4]
Muhammad Ammad-Ud-Din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. 2019. Federated collaborative filtering for privacy-preserving personalized recommendation system. arXiv preprint arXiv:1901.09888 (2019).
[5]
Naveen Farag Awad and Mayuram S. Krishnan. 2006. The personalization privacy paradox: An empirical evaluation of information transparency and the willingness to be profiled online for personalization. MIS Quart. 30, 1 (2006), 13–28. 10.2307/25148715
[6]
Borja Balle and Yu-Xiang Wang. 2018. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In Proceedings of the International Conference on Machine Learning. PMLR, 394–403.
[7]
Raef Bassily, Adam Smith, and Abhradeep Thakurta. 2014. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Proceedings of the IEEE 55th Annual Symposium on Foundations of Computer Science. IEEE, 464–473.
[8]
Mohammad Hossein Bateni, Soheil Behnezhad, Mahsa Derakhshan, Mohammad Taghi Hajiaghayi, Raimondas Kiveris, Silvio Lattanzi, and Vahab Mirrokni. 2017. Affinity clustering: Hierarchical clustering at scale. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6867–6877.
[10]
Dan Boneh, Craig Gentry, Shai Halevi, Frank Wang, and David J. Wu. 2013. Private database queries using somewhat homomorphic encryption. In Proceedings of the International Conference on Applied Cryptography and Network Security. Springer, 102–118. 10.1007/978-3-642-38980-1_7
[11]
(Leveled) Fully Homomorphic Encryption without Bootstrapping

Zvika Brakerski, Craig Gentry, Vinod Vaikuntanathan

ACM Transactions on Computation Theory 10.1145/2633600
[12]
Robin Burke. 2000. Knowledge-based recommender systems. Encyc. Libr. Inf. Syst. 69, Supplement 32 (2000), 175–186.
[13]
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Security Symposium (USENIX Security’19). 267–284.
[16]
Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. 2018. Federated meta-learning with fast convergence and efficient communication. arXiv preprint arXiv:1802.07876 (2018).
[17]
Tong Chen, Hongzhi Yin, Hongxu Chen, Rui Yan, Quoc Viet Hung Nguyen, and Xue Li. 2019. AIR: Attentional intention-aware recommender systems. In Proceedings of the IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 304–315.
[18]
Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng, Xue Li, and Xiaofang Zhou. 2020. Sequence-aware factorization machines for temporal predictive analytics. In Proceedings of the IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 1405–1416.
[20]
Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, Dimitrios Papadopoulos, and Qiang Yang. 2021. Secureboost: A lossless federated learning framework. IEEE Intell. Syst. 36, 6 (2021), 87–98.
[22]
Jill MacLaren Chorney, Abbe Marrs Garcia, Kristoffer S. Berlin, Roger Bakeman, and Zeev N. Kain. 2010. Time-window sequential analysis: An introduction for pediatric psychologists. J. Pediat. Psychol. 35, 10 (2010), 1061–1070. 10.1093/jpepsy/jsq022
[23]
Susan Craw, Ben Horsburgh, and Stewart Massie. 2015. Music recommenders: User evaluation without real users? In Proceedings of the 24th International Joint Conference on Artificial Intelligence.
[24]
Bin Dai, Ziyu Wang, and David Wipf. 2020. The usual suspects? Reassessing blame for VAE posterior collapse. In Proceedings of the International Conference on Machine Learning. PMLR, 2313–2322.
[25]
Renato Cordeiro De Amorim and Christian Hennig. 2015. Recovering the number of clusters in data sets with noise features using feature rescaling factors. Inf. Sci. 324 (2015), 126–145. 10.1016/j.ins.2015.06.039
[26]
Item-based top-Nrecommendation algorithms

Mukund Deshpande, George Karypis

ACM Transactions on Information Systems 10.1145/963770.963776
[27]
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. 2020. An efficient framework for clustered federated learning. Adv. Neural Inf. Process. Syst. 33 (2020), 19586–19597.
[29]
The MovieLens Datasets

F. Maxwell Harper, Joseph A. Konstan

ACM Transactions on Interactive Intelligent System... 10.1145/2827872
[32]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
[34]
Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. 2019. Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 573–581.
[36]
Nguyen Quoc Viet Hung, Huynh Huu Viet, Nguyen Thanh Tam, Matthias Weidlich, Hongzhi Yin, and Xiaofang Zhou. 2017. Computing crowd consensus with partial agreement. IEEE Trans. Knowl. Data Eng. 30, 1 (2017), 1–14. 10.1109/tkde.2017.2750683
[37]
Mubashir Imran, Hongzhi Yin, Tong Chen, Yingxia Shao, Xiangliang Zhang, and Xiaofang Zhou. 2020. Decentralized embedding framework for large-scale networks. In Proceedings of the International Conference on Database Systems for Advanced Applications. Springer, 425–441. 10.1007/978-3-030-59419-0_26
[38]
Nakamasa Inoue and Keita Goto. 2020. Semi-supervised contrastive learning with generalized contrastive loss and its application to speaker recognition. In Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 1641–1646.
[39]
Junjie Jia and Zhipeng Lei. 2021. Personalized recommendation algorithm for mobile based on federated matrix factorization. In Journal of Physics: Conference Series, Vol. 1802. IOP Publishing, 032021.
[40]
Zhifeng Jiang, Wei Wang, and Yang Liu. 2021. FLASHE: Additively symmetric homomorphic encryption for cross-silo federated learning. arXiv preprint arXiv:2109.00675 (2021).
[41]
Shuning Jin, Sam Wiseman, Karl Stratos, and Karen Livescu. 2020. Discrete latent variable representations for low-resource text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 4831–4842.
[43]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining (ICDM). IEEE, 197–206.
[44]
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the International Conference on Machine Learning. PMLR, 5132–5143.
[46]
Muah Kim, Onur Günlü, and Rafael F. Schaefer. 2021. Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2650–2654.
[47]
Jakub Konečnỳ, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).
[48]
Pedro Giovanni Leon, Blase Ur, Yang Wang, Manya Sleeper, Rebecca Balebako, Richard Shay, Lujo Bauer, Mihai Christodorescu, and Lorrie Faith Cranor. 2013. What matters to users? Factors that affect users’ willingness to share information with online advertisers. In Proceedings of the 9th Symposium on Usable Privacy and Security. 1–12.

Showing 50 of 103 references

Cited By
93
Communication-Efficient Federated Neural Collaborative Filtering with Multi-Armed Bandits

WAQAR ALI, Muhammad Ammad-Ud-Din · 2025

ACM Transactions on Recommender Sys...
Federated cross-domain recommendation system based on bias eliminator and personalized extractor

Yicheng Di, Hongjian Shi · 2024

Knowledge and Information Systems
Metrics
93
Citations
103
References
Details
Published
Feb 07, 2023
Vol/Issue
41(3)
Pages
1-30
License
View
Authors
M
Mubashir Imran

The University of Queensland, Brisbane, QLD, Australia

H
Hongzhi Yin

The University of Queensland, Brisbane, QLD, Australia

Q
Quoc Viet Hung Nguyen

Griffith University, Gold Coast, Australia

A
Alexander Zhou

Hong Kong University of Science and Technology, Hong Kong, China

K
Kai Zheng

University of Electronic Science and Technology of China, Chengdu, Sichuan, China

Funding
The University of Queensland Award: NS-2103
Discovery Project Award: DP190101985
Australian Research Council’s Future Fellowship Award: FT210100624
Discovery Early Career Research Award Award: DE200101465
Cite This Article
Mubashir Imran, Hongzhi Yin, Quoc Viet Hung Nguyen, et al. (2023). ReFRS: Resource-efficient Federated Recommender System for Dynamic and Diversified User Preferences. ACM Transactions on Information Systems, 41(3), 1-30. https://doi.org/10.1145/3560486
Related

You May Also Like

Evaluating collaborative filtering recommender systems

Jonathan L. Herlocker, Joseph A. Konstan · 2004

4,119 citations

Cumulated gain-based evaluation of IR techniques

Kalervo Järvelin, Jaana Kekäläinen · 2002

3,332 citations

Item-based top-Nrecommendation algorithms

Mukund Deshpande, George Karypis · 2004

1,700 citations

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Lei Huang, Weijiang Yu · 2025

1,105 citations

Latent semantic models for collaborative filtering

Thomas Hofmann · 2004

987 citations