TY - GEN
T1 - Semi-Decentralized Federated Edge Learning for Fast Convergence on Non-IID Data
AU - Sun, Yuchang
AU - Shao, Jiawei
AU - Mao, Yuyi
AU - Wang, Jessie Hui
AU - Zhang, Jun
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022/4
Y1 - 2022/4
N2 - Federated edge learning (FEEL) has emerged as an effective approach to reduce the large communication latency in Cloud-based machine learning solutions, while preserving data privacy. Unfortunately, the learning performance of FEEL may be compromised due to limited training data in a single edge cluster. In this paper, we investigate a novel framework of FEEL, namely semi-decentralized federated edge learning (SD-FEEL). By allowing model aggregation across different edge clusters, SD-FEEL enjoys the benefit of FEEL in reducing the training latency, while improving the learning performance by accessing richer training data from multiple edge clusters. A training algorithm for SD-FEEL with three main procedures in each round is presented, including local model updates, intra-cluster and inter-cluster model aggregations, which is proved to converge on non-independent and identically distributed (non-IID) data. We also characterize the interplay between the network topology of the edge servers and the communication overhead of inter-cluster model aggregation on the training performance. Experiment results corroborate our analysis and demonstrate the effectiveness of SD-FFEL in achieving faster convergence than traditional federated learning architectures. Besides, guidelines on choosing critical hyper-parameters of the training algorithm are also provided.
AB - Federated edge learning (FEEL) has emerged as an effective approach to reduce the large communication latency in Cloud-based machine learning solutions, while preserving data privacy. Unfortunately, the learning performance of FEEL may be compromised due to limited training data in a single edge cluster. In this paper, we investigate a novel framework of FEEL, namely semi-decentralized federated edge learning (SD-FEEL). By allowing model aggregation across different edge clusters, SD-FEEL enjoys the benefit of FEEL in reducing the training latency, while improving the learning performance by accessing richer training data from multiple edge clusters. A training algorithm for SD-FEEL with three main procedures in each round is presented, including local model updates, intra-cluster and inter-cluster model aggregations, which is proved to converge on non-independent and identically distributed (non-IID) data. We also characterize the interplay between the network topology of the edge servers and the communication overhead of inter-cluster model aggregation on the training performance. Experiment results corroborate our analysis and demonstrate the effectiveness of SD-FFEL in achieving faster convergence than traditional federated learning architectures. Besides, guidelines on choosing critical hyper-parameters of the training algorithm are also provided.
KW - communication efficiency
KW - distributed machine learning
KW - Federated learning
KW - non-IID data
UR - http://www.scopus.com/inward/record.url?scp=85130736390&partnerID=8YFLogxK
U2 - 10.1109/WCNC51071.2022.9771904
DO - 10.1109/WCNC51071.2022.9771904
M3 - Conference article published in proceeding or book
AN - SCOPUS:85130736390
T3 - IEEE Wireless Communications and Networking Conference, WCNC
SP - 1898
EP - 1903
BT - 2022 IEEE Wireless Communications and Networking Conference, WCNC 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE Wireless Communications and Networking Conference, WCNC 2022
Y2 - 10 April 2022 through 13 April 2022
ER -