Split Federated Learning on Micro-controllers A Keyword Spotting Showcase

2025-05-03 0 0 1.12MB 5 页 10玖币

侵权投诉

Split Federated Learning on Micro-controllers: A Keyword

Spoing Showcase

Jingtao Li

ASU

Tempe, USA

Runcong Kuang

ASU

Tempe, USA

ABSTRACT

Nowadays, AI companies improve service quality by aggressively

collecting users’ data generated by edge devices, which jeopardizes

data privacy. To prevent this, Federated Learning is proposed as a

private learning scheme, using which users can locally train the

model without collecting users’ raw data to servers. However, for

machine-learning applications on edge devices that have hard mem-

ory constraints, implementing a large model using FL is infeasible.

To meet the memory requirement, a recent collaborative learning

scheme named split federal learning is a potential solution since it

keeps a small model on the device and keeps the rest of the model

on the server. In this work, we implement a simply SFL framework

on the Arduino board and verify its correctness on the Chinese

digits audio dataset for keyword spotting application with over 90%

accuracy. Furthermore, on the English digits audio dataset, our SFL

implementation achieves 13.89% higher accuracy compared to a

state-of-the-art FL implementation.

KEYWORDS

TinyML, Federated Learning, Keyword Spotting

ACM Reference Format:

Jingtao Li and Runcong Kuang. 2022. Split Federated Learning on Micro-

controllers: A Keyword Spotting Showcase. In Proceedings of Make sure to

enter the correct conference title from your rights conrmation emai (Con-

ference acronym ’XX). ACM, New York, NY, USA, 5 pages. https://doi.org/

XXXXXXX.XXXXXXX

1 INTRODUCTION

Increasing accessibility of edge computing devices has shown great

impact on our daily life, and at the same time, their generated

data powers up machine learning applications such as computer

vision and voice recognition. However, as these devices are close

to our living environment, it starts to raise concern on the data

privacy. General Data Protection Regulation (GDPR) that is recently

approved by European Union imposes a strong limitation on col-

lecting data from users. In this context, how to utilize these valuable

data while comply with GDPR is still a challenge for AI community.

In fact, private schemes such as multi-party computation (MPC)[

]

and homomorphic encryption (HE) can enable provable privacy

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specic permission and/or a

fee. Request permissions from permissions@acm.org.

Conference acronym ’XX, June 03–05, 2022, Woodstock, NY

ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00

https://doi.org/XXXXXXX.XXXXXXX

while handling users’ data. However, the large overhead of computa-

tion and communication of these private methods are not practical

for public use (though they can be good for cross-silo use case).

Thus, companies start to investigate cheap but privacy-preserving

learning schemes. Google presents Federated Learning (FL) [

] as

one of these schemes and deploy it in its Google assistant and

Google Keyboard applications.

The key idea of FL is to prevent users’ data from leaving the

device. In its design, each user acquires a local copy of the model,

train it on user’s own private data, and send the updated parameters

to the server periodically. Server, once collecting updated parame-

ters from all users, perform an average (as in FedAvg[

]) to get the

global model updated and transfer to user for training in the next

round. For the rst time, [

] brings the Federated Learning idea to

the TinyML community, where they are able to demonstrate the

successful implementation of FL in a tiny micro-controller for a

2-layer MLP model in a binary keyword classication problem.

A clear limitation of using FL is that it requires user have the

ability to train the entire model locally on the device. For embedded

systems which are extremely constrained by memory and power,

training locally is not possible for an industry-scale model. Thus,

[

] uses a tiny 2-layer MLP of 16K parameter, mostly in the concern

of the limited SRAM of the micro-controller.

For more complex datasets and use cases, model has to be larger.

Thus, we investigate the realization of Split Federated Learning

(SFL) [

] as an edge-friendly version of FL on a micro-controller.

SFL, splits the model into two parts, client-side model and server-

side model. Client-side model is designed to be small enough to run

on the device and server-side model can be very large to achieve a

good accuracy. As a split version of Federated Learning, the raw

data is also protected in SFL since it is processed pure in local

devices. Moreover, SFL allows users to have only a small part of

the model which can relieve the hardware requirement.

In the scope of our project, we implemented SFL on Arduino

Nano BLE 33 sense for a multi-class keyword classication task,

where an audio dataset of digits pronounced in Chinese and English

is classied respectively. By SFL, We are able to use very large

model in training because server does not have memory limitation

and has fast training speed. We compared dierent model sizes

(FL’s and SFL’s) in terms of the model’s classication accuracy,

and gave a proof-of-concept example of using SFL can achieve

better performance than using FL in practice. We also demonstrated

that SFL enables using bigger model to enhance the accuracy. SFL

shows 13.89% better accuracy using a large CNN model compared

to Federated Learning where only a small MLP can be used. After

training, we compressed the entire model and put it back to the

device. Then, the device can perform inference without dependence

arXiv:2210.01961v1 [cs.LG] 4 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

SplitFederatedLearningonMicro-controllers:AKeywordSpottingShowcaseJingtaoLiASUTempe,USARuncongKuangASUTempe,USAABSTRACTNowadays,AIcompaniesimproveservicequalitybyaggressivelycollectingusers’datageneratedbyedgedevices,whichjeopardizesdataprivacy.Topreventthis,FederatedLearningisproposedasaprivatelear...

展开>> 收起<<

Split Federated Learning on Micro-controllers A Keyword Spotting Showcase.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Split Federated Learning on Micro-controllers A Keyword Spotting Showcase

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: