Improving the Robustness of Federated Learning by Sharing Statistical Characteristics

Federated Learning (FL) stands as a pioneering paradigm that not only amplifies the generalization capabilities of machine learning models but also upholds the sanctity of user privacy through its distributed learning approach [1]. Nevertheless, it is imperative to acknowledge a fundamental challenge inherent to this paradigm—namely, the absence of a priori knowledge concerning data distribution across participating nodes[2]. 

While ensuring user privacy, if only statistical characteristics such as nth-order moments of the dataset are shared, rather than the data, it is possible to on reconstruct the distribution of the data. These statistical characteristics, can (i) identify whether the shared model has been tampered with, (ii) whether the model is locally applicable. Therefore, this student project wants to share the statistical characteristics of the data from each node in a FL network, which in turn achieves the goal of improving the robustness of DFL. Hence, the central objective of this student project is to facilitate the exchange of statistical characteristics derived from the data residing at each individual node within a DFL network. By prioritizing the sharing of these statistical insights, rather than the raw data, the project endeavors to strike a delicate balance between data privacy and collaborative model enhancement. This approach, in essence, contributes significantly to bolstering the robustness and effectiveness of DFL systems. 

[1] Beltrán, E. T. M., Pérez, M. Q., Sánchez, P. M. S., Bernal, S. L., Bovet, G., Pérez, M. G., ... & Celdrán, A. H. (2022). Decentralized federated learning: Fundamentals, state-of-the-art, frameworks, trends, and challenges. arXiv preprint arXiv:2211.08413.

[2] Ma, Xiaodong, et al. "A state-of-the-art survey on solving non-IID data in Federated Learning." Future Generation Computer Systems 135 (2022): 244-258.

[3] https://webapp.sebgroup.com/mb/mblib.nsf/alldocsbyunid/96051B45DE874642C12586B9005020CD/$FILE/entropy_pooling.pdf