hFedLAP: A Hybrid Federated Learning to Enhance Peer-to-Peer
Received: 23 March 2024 | Revised: 22 April 2024 | Accepted: 23 April 2024 | Online: 1 June 2024
Corresponding author: Ismail Elshair
Abstract
The concept of Federated Learning (FL) is a branch of Machine Learning (ML) that enables localized training of models without transferring data from local devices to a central server. FL can be categorized into two main topologies: Aggregation Server Topology (AST) and Peer-to-Peer (P2P). While FL offers advantages in terms of data privacy and decentralization, it also exhibits certain limitations in efficiency and bottleneck. However, the P2P topology does not require a server and allows only for a small number of devices. To overcome these limitations, this study proposes a hybrid FL Aggregation of P2P (hFedLAP) that mitigates some of the limitations of AST by combining it with P2P. This fusion model helps to remove the bottleneck and combines the advantages of both topologies. In the proposed hFedLAP model, clients are organized into 49 groups, each consisting of 51 clients, including one in each group serving as a client and an admin node in a P2P setup. In these groups, communication is restricted to admin nodes, supporting a maximum of 2,495 devices. Platform accuracy is maintained by implementing measures to prevent new devices with inadequate accuracy levels from joining until they attain the minimum required accuracy. The experimental results of hFedLAP were compared with AST and P2P using the MNIST dataset, showing that hFedLAP outperformed AST and P2P, achieving remarkable accuracy and scalability, with accuracy levels reaching 98.81%.
Keywords:
hybrid model, federated learning, aggregation server, AST FL, peer-to-peer FL, machine learning, hybrid FL modelDownloads
References
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, "Communication-Efficient Learning of Deep Networks from Decentralized Data," in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, Apr. 2017, pp. 1273–1282.
H. A. Owida, A. Al-Ghraibah, and M. Altayeb, "Classification of Chest X-Ray Images using Wavelet and MFCC Features and Support Vector Machine Classifier," Engineering, Technology & Applied Science Research, vol. 11, no. 4, pp. 7296–7301, Aug. 2021.
B. K. Ponukumati, P. Sinha, M. K. Maharana, A. V. P. Kumar, and A. Karthik, "An Intelligent Fault Detection and Classification Scheme for Distribution Lines Using Machine Learning," Engineering, Technology & Applied Science Research, vol. 12, no. 4, pp. 8972–8977, Aug. 2022.
W. Li et al., "Privacy-Preserving Federated Brain Tumour Segmentation," in Machine Learning in Medical Imaging, Shenzhen, China, 2019, pp. 133–141.
Q. Yang, Y. Liu, T. Chen, and Y. Tong, "Federated Machine Learning: Concept and Applications," ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, Jan. 2019, Art. no. 12.
Q. Li et al., "A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection," IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3347–3366, Apr. 2023.
P. Kairouz et al., "Advances and Open Problems in Federated Learning," Foundations and Trends® in Machine Learning, vol. 14, no. 1-2, pp. 1–210, Jun. 2021.
A. Hard et al., "Federated Learning for Mobile Keyboard Prediction." arXiv, Feb. 28, 2019.
X. Ma, J. Zhang, S. Guo, and W. Xu, "Layer-wised Model Aggregation for Personalized Federated Learning," in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, Jun. 2022, pp. 10082–10091.
J. Goetz, K. Malik, D. Bui, S. Moon, H. Liu, and A. Kumar, "Active Federated Learning." arXiv, Sep. 27, 2019.
H. Wang, M. Yurochkin, Y. Sun, D. Papailiopoulos, and Y. Khazaeni, "Federated Learning with Matched Averaging." arXiv, Feb. 15, 2020.
N. Mhaisen, A. A. Abdellatif, A. Mohamed, A. Erbad, and M. Guizani, "Optimal User-Edge Assignment in Hierarchical Federated Learning Based on Statistical Properties and Network Topology Constraints," IEEE Transactions on Network Science and Engineering, vol. 9, no. 1, pp. 55–66, Jan. 2022.
L. Liu, J. Zhang, S. H. Song, and K. B. Letaief, "Client-Edge-Cloud Hierarchical Federated Learning," in ICC 2020 - 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, Jun. 2020, pp. 1–6.
S. Hosseinalipour et al., "Multi-Stage Hybrid Federated Learning Over Large-Scale D2D-Enabled Fog Networks," IEEE/ACM Transactions on Networking, vol. 30, no. 4, pp. 1569–1584, Aug. 2022.
Y. Zhao et al., "Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices," IEEE Internet of Things Journal, vol. 8, no. 3, pp. 1817–1829, Oct. 2021.
A. G. Roy, S. Siddiqui, S. Pölsterl, N. Navab, and C. Wachinger, "BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning." arXiv, May 16, 2019.
J. Lee, J. Oh, S. Lim, S. Y. Yun, and J. G. Lee, "TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture." arXiv, Oct. 01, 2021.
Z. Wang, Y. Hu, J. Xiao, and C. Wu, "Efficient Ring-topology Decentralized Federated Learning with Deep Generative Models for Industrial Artificial Intelligent." arXiv, Apr. 15, 2021.
O. Marfoq, C. Xu, G. Neglia, and R. Vidal, "Throughput-Optimal Topology Design for Cross-Silo Federated Learning," in Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 19478–19487.
Q. Li, B. He, and D. Song, "Practical One-Shot Federated Learning for Cross-Silo Setting." arXiv, May 20, 2021.
A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, "An Efficient Framework for Clustered Federated Learning," IEEE Transactions on Information Theory, vol. 68, no. 12, pp. 8076–8091, Sep. 2022.
M. M. Yaqoob, M. Nazir, M. A. Khan, S. Qureshi, and A. Al-Rasheed, "Hybrid Classifier-Based Federated Learning in Health Service Providers for Cardiovascular Disease Prediction," Applied Sciences, vol. 13, no. 3, Jan. 2023.
G. Li et al., "FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity," in Proceedings of the 51st International Conference on Parallel Processing, Bordeaux, France, Sep. 2022, pp. 1–11.
S. Hosseinalipour et al., "Multi-Stage Hybrid Federated Learning Over Large-Scale D2D-Enabled Fog Networks," IEEE/ACM Transactions on Networking, vol. 30, no. 4, pp. 1569–1584, Dec. 2022.
M. Cao, Y. Zhang, Z. Ma, and M. Zhao, "C2S: Class-aware client selection for effective aggregation in federated learning," High-Confidence Computing, vol. 2, no. 3, Sep. 2022, Art. no. 100068.
S. Wang et al., "Adaptive Federated Learning in Resource Constrained Edge Computing Systems," IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1205–1221, Mar. 2019.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
Downloads
How to Cite
License
Copyright (c) 2024 Ismail Elshair, Tariq J. S. Khanzada
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
