nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Rethinking Personalized Federated Learning with Clustering-Based Dynamic Graph Propagation

verfasst von : Jiaqi Wang, Yuzhong Chen, Yuhang Wu, Mahashweta Das, Hao Yang, Fenglong Ma

Erschienen in: Advances in Knowledge Discovery and Data Mining

Verlag: Springer Nature Singapore

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Most existing personalized federated learning approaches are based on intricate designs, which often require complex implementation and tuning. In order to address this limitation, we propose a simple yet effective personalized federated learning framework. Specifically, during each communication round, we group clients into multiple clusters based on their model training status and data distribution on the server side. We then consider each cluster center as a node equipped with model parameters and construct a graph that connects these nodes using weighted edges. Additionally, we update the model parameters at each node by propagating information across the entire graph. Subsequently, we design a precise personalized model distribution strategy to allow clients to obtain the most suitable model from the server side. We conduct experiments on three image benchmark datasets and create synthetic structured datasets with three types of typologies. Experimental results demonstrate the effectiveness of the proposed FedCedar.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Probabilistic Guarantees of Stochastic Recursive Gradient in Non-convex Finite Sum Problems

Nächstes Kapitel Unveiling Backdoor Risks Brought by Foundation Models in Heterogeneous Federated Learning

Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. arXiv preprint arXiv:1912.00818 (2019)

Briggs, C., Fan, Z., Andras, P.: Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In: 2020 International Joint Conference on Neural Networks, pp. 1–9. IEEE (2020)

Chen, F., Long, G., Wu, Z., Zhou, T., Jiang, J.: Personalized federated learning with graph. arXiv preprint arXiv:2203.00829 (2022)

Fallah, A., Mokhtari, A., Ozdaglar, A.: Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. In: Advances in Neural Information Processing Systems, vol. 33, pp. 3557–3568 (2020)

Ghosh, A., Chung, J., Yin, D., Ramchandran, K.: An efficient framework for clustered federated learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 19586–19597 (2020)

He, C., Ceyani, E., Balasubramanian, K., Annavaram, M., Avestimehr, S.: SpreadGNN: serverless multi-task federated learning for graph neural networks. arXiv preprint arXiv:2106.02743 (2021)

Hu, K., Wu, J., Li, Y., Lu, M., Weng, L., Xia, M.: FedGCN: federated learning-based graph convolutional networks for non-Euclidean spatial data. Mathematics 10(6), 1000 (2022)CrossRef

Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)CrossRef

Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)

10.

Li, X., Wu, C., Wang, J.: Unveiling backdoor risks brought by foundation models in heterogeneous federated learning. arXiv preprint arXiv:2311.18350 (2023)

11.

Lin, S., Han, Y., Li, X., Zhang, Z.: Personalized federated learning towards communication efficiency, robustness and fairness. In: Advances in Neural Information Processing Systems (2022)

12.

Liu, C.T., Wang, C.Y., Chien, S.Y., Lai, S.H.: FedFR: joint optimization federated framework for generic and personalized face recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 1656–1664 (2022)

13.

Liu, Z., et al.: Contribution-aware federated learning for smart healthcare. In: Proceedings of the 34th Annual Conference on Innovative Applications of Artificial Intelligence (2022)

14.

Long, G., Tan, Y., Jiang, J., Zhang, C.: Federated learning for open banking. In: Yang, Q., Fan, L., Yu, H. (eds.) Federated Learning. LNCS (LNAI), vol. 12500, pp. 240–254. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63076-8_17CrossRef

15.

Long, G., Xie, M., Shen, T., Zhou, T., Wang, X., Jiang, J.: Multi-center federated learning: clients clustering for better personalization. In: World Wide Web, pp. 1–20 (2022)

16.

Long, Z., Wang, J., Wang, Y., Xiao, H., Ma, F.: FedCON: a contrastive framework for federated semi-supervised learning. arXiv preprint arXiv:2109.04533 (2021)

17.

Lou, G., Liu, Y., Zhang, T., Zheng, X.: STFL: a temporal-spatial federated learning framework for graph neural networks. arXiv preprint arXiv:2111.06750 (2021)

18.

Ma, Z., Lu, Y., Li, W., Cui, S.: Beyond random selection: a perspective from model inversion in personalized federated learning. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) ECML PKDD 2022. LNCS, vol. 13716, pp. 572–586. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-26412-2_35CrossRef

19.

Marchand, T., Muzellec, B., Beguier, C., Terrail, J.O.d., Andreux, M.: Securefedyj: a safe feature Gaussianization protocol for federated learning. arXiv preprint arXiv:2210.01639 (2022)

20.

McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)

21.

Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66(336), 846–850 (1971)CrossRef

22.

Solanki, S., Kanaparthy, S., Damle, S., Gujar, S.: Differentially private federated combinatorial bandits with constraints. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) Machine Learning and Knowledge Discovery in Databases - ECML PKDD 2022. Lecture Notes in Computer Science, vol. 13716, pp. 620–637. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-26412-2_38CrossRef

23.

Dinh, C.T., Tran, N., Nguyen. J.: Personalized federated learning with Moreau envelopes. In: Advances in Neural Information Processing Systems, vol. 33, pp. 21394–21405 (2020)

24.

Wang, J., Ma, F.: Federated learning for rare disease detection: a survey. Rare Disease Orphan Drugs J. 2, 1–14 (2023)CrossRef

25.

Wang, J., Qian, C., Cui, S., Glass, L., Ma, F.: Towards federated COVID-19 vaccine side effect prediction. In: Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds.) Machine Learning and Knowledge Discovery in Databases - ECML PKDD 2022. Lecture Notes in Computer Science, vol. 13718, pp. 437–452. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-26422-1_27CrossRef

26.

Wang, J., et al.: Towards personalized federated learning via heterogeneous model reassembly. In: Advances in Neural Information Processing Systems (2023)

27.

Wang, J., Zeng, S., Long, Z., Wang, Y., Xiao, H., Ma, F.: Knowledge-enhanced semi-supervised federated learning for aggregating heterogeneous lightweight clients in IoT. In: Proceedings of the 2023 SIAM International Conference on Data Mining, pp. 496–504. SIAM (2023)

28.

Wu, C., Li, X., Wang, J.: Vulnerabilities of foundation model integrated federated learning under adversarial threats. arXiv preprint arXiv:2401.10375 (2024)

29.

Wu, C., Wu, F., Cao, Y., Huang, Y., Xie, X.: FedGNN: federated graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2102.04925 (2021)

30.

Xie, M., et al.: Multi-center federated learning. arXiv preprint arXiv:2005.01026 (2020)

31.

Zhang, K., Yang, C., Li, X., Sun, L., Yiu, S.M.: Subgraph federated learning with missing neighbor generation. In: Advances in Neural Information Processing Systems, vol. 34, pp. 6671–6682 (2021)

32.

Zhang, X., Li, Y., Li, W., Guo, K., Shao, Y.: Personalized federated learning via variational Bayesian inference. In: International Conference on Machine Learning, pp. 26293–26310. PMLR (2022)

Titel: Rethinking Personalized Federated Learning with Clustering-Based Dynamic Graph Propagation
verfasst von: Jiaqi Wang
Yuzhong Chen
Yuhang Wu
Mahashweta Das
Hao Yang
Fenglong Ma
Verlag: Springer Nature Singapore
Buch: Advances in Knowledge Discovery and Data Mining
Print ISBN: 978-981-9722-61-7

Electronic ISBN: 978-981-9722-59-4

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-981-97-2259-4_12

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner