Authors : Saifuddin Shaik Mohammed

Volume/Issue : Volume 10 - 2025, Issue 9 - September

Google Scholar : https://tinyurl.com/ytzmrevn

Scribd : https://tinyurl.com/mt23kath

DOI : https://doi.org/10.38124/ijisrt/25sep1145

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Note : Google Scholar may take 30 to 40 days to display the article.

Abstract : The rapid progress of large-scale models, including foundational and generative, brings to the forefront the tension between data-driven innovation and core privacy concerns. Such contracts as the GDPR and the undue privacy threats of data aggregation make centralized training approaches less desirable. To analyze the data’s distributed characteristics and their application to FLO, we investigate the role of federation analytics in a plausible paradigm that shunts data. In this paper, we present a new federated learning (FL) framework enhanced with cutting-edge privacy technologies (PET) such as Differential privacy for user-level formal guarantees of confidentiality, and strengthened secure Multi-Party Computation (SMPC), which guards the model updates. This paper studies more recent approaches to resolving the principal challenges of FL: statistical heterogeneity, communication bottlenecks, and vulnerability to adversarial attacks. We greatly appreciate what this new method portends, especially for training large language models (LLMs) and the more delicate areas of healthcare and finance. By evaluating certain existing limitations, such as the complexities of federated fine- tuning and model fairness, it is clear that an architecture with exemplary performance in FL serves as a model for scalable, secure, and privacy cop.

Keywords : Federated Learning, Data Privacy, Large Language Models (LLMs), Decentralized AI, Differential Privacy, Secure Aggregation, Statistical Heterogeneity.

References :

1. Zhao Y, Chen Z, Liu Z, et al. A Survey on Federated Learning for Foundation Models. arXiv preprint arXiv:2403.09392. 2024 Mar.
2. European Parliament and Council of the European Union. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union. 2016; L119:1-88.
3. Rocher L, Hendrickx JM, de Montjoye Y-A. Estimating the success of re-identifications in incomplete datasets using generative models. Nature Communications. 2021;12(1):3069.
4. Yang Q, Liu Y, Cheng Y, Kang Y, Chen T, Yu H. Federated Learning. 2nd ed. Morgan & Claypool Publishers; 2024.
5. Cui H, Ruan W, Li S. FedFair: A Survey on Fairness in Federated Learning. arXiv preprint arXiv:2302.13575. 2023 Feb.
6. Karimireddy SP, Kale S, Mohri M, et al. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In: Proceedings of the 37th International Conference on Machine Learning (ICML); 2020. p. 5132-5143. [Note: Foundational but heavily cited in all 2022-2024 literature on the topic].
7. Marfoq O, Neglia G, Bellet A, et al. Personalized Federated Learning: A Meta-Learning Approach. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(4):4371-4388.
8. Che C, Wang Z, Wang X, et al. A Survey on Backdoor Attacks and Defenses in Federated Learning. arXiv preprint arXiv:2401.12381. 2024 Jan.
9. [9] Li X, Wang L, Liu B, et al. A Survey on Federated Learning with Differential Privacy: Advances and Applications. ACM Computing Surveys. 2023;56(5):1-39.
10. Thapa C, Chamikara MA, Camtepe S. A Comprehensive Survey on the Role of Differential Privacy in Federated Learning. ACM Computing Surveys. 2022;55(8):1-38.
11. He K, Liu Y, Li J, et al. Communication-Efficient and Privacy-Preserving Federated Learning for Industrial IoT. IEEE Transactions on Industrial Informatics. 2021;18(5):3476-3485.
12. Li X, Huang K, Yang W, et al. On the Convergence of FedAvg on Non-IID Data. In: Proceedings of the 9th International Conference on Learning Representations (ICLR); 2021.
13. Haddadpour F, Kamani MM, Mokhtari A, et al. Federated Learning with Communication-Efficient Local Adam. IEEE Transactions on Signal Processing. 2021;69:5589-5604.
14. Fan L, Zhang X, Wang W, et al. A Survey of Federated Learning for Large Language Models. arXiv preprint arXiv:2311.13247. 2023 Nov.
15. Bao J, Wu C, Zhang Z, et al. FedLoRA: Federated Low-Rank Adaptation of Large Language Models. arXiv preprint arXiv:2403.08947. 2024 Mar.
16. Florescu LM, Busa-Fekete R, Gummadi KP, et al. Federated Learning in the Medical Domain: A Systematic Literature Review. ACM Transactions on Computing for Healthcare. 2024;5(2):1-41.
17. Meng Y, Liang G, Li Y, et al. A Survey on Federated Learning for Financial Crime Detection. IEEE Transactions on Knowledge and Data Engineering. 2023;36(3):1314-1335.
18. Ezzeldin Y, Yan S, He C, et al. FairFed: A Bias-Aware Fair Federated Learning Framework. Proceedings of the ACM on Human-Computer Interaction. 2023;7(CSCW1):1-26.
19. Bourtoule L, Chandrasekaran V, Choquette-Choo CA, et al. Machine Unlearning. In: 2021 IEEE Symposium on Security and Privacy (SP); 2021. p. 141-159.
20. Nguyen DC, Ding M, Pathirana PN, et al. Federated Learning for Industrial Internet of Things: A Comprehensive Survey. IEEE Communications Surveys & Tutorials. 2021;23(3):1622-1658.
21. Karimireddy SP, Kale S, Mohri M, et al. SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. In: Proceedings of the 37th International Conference on Machine Learning (ICML); 2020.
22. Marfoq O, Neglia G, Bellet A, et al. Personalized Federated Learning: A Meta-Learning Approach. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(4):4371-4388.
23. Li X, Wang L, Liu B, et al. A Survey on Federated Learning with Differential Privacy: Advances and Applications. ACM Computing Surveys. 2023;56(5):1-39.
24. He K, Liu Y, Li J, et al. Communication-Efficient and Privacy-Preserving Federated Learning for Industrial IoT. IEEE Transactions on Industrial Informatics. 2021;18(5):3476-3485.
25. Quintana, G. I., et al. (2024). BN-SCAFFOLD: controlling the drift of Batch Normalization statistics in Federated Learning. arXiv:2410.03281.
26. Behnia, R., et al. (2024). Efficient Secure Aggregation for Privacy-Preserving Federated Machine Learning. ACSAC 2024.
27. Li X, Huang K, Yang W, et al. On the Convergence of FedAvg on Non-IID Data. In: Proceedings of the 9th International Conference on Learning Representations (ICLR); 2021.
28. Cheng, Z., Huang, X., Wu, P., & Yuan, K. (2024). Momentum Benefits Non-IID Federated Learning Simply and Provably. ICML 2024.
29. Huang, X., Li, P., & Li, X. (2024). Stochastic Controlled Averaging for Federated Learning with Communication Compression. ICLR 2024.