Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning Architectures in a Non-IID Scenario

2025-04-30 0 0 845.01KB 15 页 10玖币

Analyzing the Robustness of Decentralized Horizontal and Vertical

Federated Learning Architectures in a Non-IID Scenario

Pedro Miguel Sánchez Sáncheza,∗,Alberto Huertas Celdránb,Enrique Tomás Martínez Beltrána,

Daniel Demeterb,Gérôme Bovetc,Gregorio Martínez Pérezaand Burkhard Stillerb

aDepartment of Information and Communications Engineering, University of Murcia, Murcia 30100, Spain. Corresponding author: Pedro Miguel

Sanchez (pedromiguel.sanchez@um.es)

bCommunication Systems Group (CSG), Department of Informatics (IfI), University of Zurich UZH, 8050 Zürich, Switzerland

cCyber-Defence Campus within armasuisse Science & Technology, CH—3602 Thun, Switzerland

ARTICLE INFO

Keywords:

Horizontal Federated Learning

Vertical Federated Learning

Adversarial attacks

Non-IID Data

Distributed Artiﬁcial Intelligence

Robustness.

ABSTRACT

Federated learning (FL) allows participants to collaboratively train machine and deep learning

models while protecting data privacy. However, the FL paradigm still presents drawbacks af-

fecting its trustworthiness since malicious participants could launch adversarial attacks against

the training process. Related work has studied the robustness of horizontal FL scenarios under

diﬀerent attacks. However, there is a lack of work evaluating the robustness of decentralized

vertical FL and comparing it with horizontal FL architectures aﬀected by adversarial attacks.

Thus, this work proposes three decentralized FL architectures, one for horizontal and two for

vertical scenarios, namely HoriChain, VertiChain, and VertiComb. These architectures present

diﬀerent neural networks and training protocols suitable for horizontal and vertical scenarios.

Then, a decentralized, privacy-preserving, and federated use case with non-IID data to classify

handwritten digits is deployed to evaluate the performance of the three architectures. Finally, a

set of experiments computes and compares the robustness of the proposed architectures when

they are aﬀected by diﬀerent data poisoning based on image watermarks and gradient poisoning

adversarial attacks. The experiments show that even though particular conﬁgurations of both

attacks can destroy the classiﬁcation performance of the architectures, HoriChain is the most

robust one.

1. Introduction

Communication and computing evolution have brought to reality new paradigms, such as the Internet-of-Things

(IoT), generating massive amounts of decentralized and heterogeneous data. This trend has given machine and deep

learning (ML/DL) techniques huge relevance in our current society since data quantity and quality are essential re-

quirements to operate successfully. However, with stakeholders becoming more aware of how their data is used and

data privacy ﬁnding its way into the mainstream debate, a new approach to ML/DL was required to appease these con-

cerns. Introduced by Google in 2016 [1], federated learning (FL) emerged as a possible solution. FL is a paradigm that

enables participants to train an ML/DL model collaboratively, and most importantly, it does so without having to share

the participants’ datasets. The paradigm provides three diﬀerent scenarios according to how data is distributed between

participants. In horizontal federated learning (HFL), participants (also known as clients) have the same feature space

but diﬀerent samples [2]. In the vertical federated learning (VFL) scenario, participants hold diﬀerent features, but

the same samples [3]. Finally, when neither feature space nor samples are the same between participants, the scenario

is termed federated transfer learning (FTL). In recent years, heterogeneous use cases suitable for the previous three

scenarios, especially for HFL, have provided levels of performance comparable to classical ML/DL algorithms where

data privacy is not considered [4]. Furthermore, since the datasets never leave the participants’ possession in the FL

paradigm, the logistical problems of aggregating, storing, and maintaining data in central silos are eliminated.

Despite the advantages of FL, the decentralized nature of its training phase exposes the previous three scenarios to

new attack surfaces [5]. In particular, malicious participants executing adversarial attacks to aﬀect the trustworthiness

of FL models are one of the most representative cybersecurity concerns of this paradigm. More in detail, malicious

participants could join the federation to disrupt, corrupt, or delay the model learning process [6]. Another well-known

target of adversaries is to infer sensitive information from other participants, but it is out of the scope of this work [7].

ORCID(s): 0000-0002-6444-2102 (P.M. Sánchez Sánchez); 0000-0001-7125-1710 (A. Huertas Celdrán); 0000-0002-5169-2815

(E.T. Martínez Beltrán); 0000-0002-4534-3483 (G. Bovet); 0000-0001-5532-6604 (G. Martínez Pérez); 0000-0002-7461-7463 (B. Stiller)

First Author et al.: Preprint submitted to Elsevier Page 1 of 15

arXiv:2210.11061v1 [cs.LG] 20 Oct 2022

Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning

To destroy the global model performance, the literature has documented several data and model falsiﬁcation attacks

consisting of poisoning data, labels, or weights during training [8]. The detection and mitigation of these attacks

are challenging tasks since there is a trade-oﬀ between the performance of the global model and the privacy of the

participants’ sensitive data. In other words, since the FL paradigm aims to expose as little information about the

individual participants’ data as possible, recognizing and mitigating the presence of poisoned data samples is not easy

[9]. Therefore, despite the existing detection and mitigation solutions, such as the usage of clustering techniques to

detect anomalies in model parameters [10] or the use of secure aggregation functions to remove noisy weights [11],

no robust solution exists nowadays.

Additionally, before thinking about detecting and mitigating adversarial attacks, it is critical to analyze the impact

of heterogeneous attacks on diﬀerent FL scenarios. In this sense, robust FL architectures and models should be built

to collaborate with detection and mitigation techniques and reduce attack impacts as much as possible. However,

the following challenges are still open regarding FL architectural robustness. First, the impact of existing data and

model poisoning attacks has mainly been validated in horizontal scenarios, being decentralized vertical scenarios

unexplored. Second, while diﬀerent categories of attacks are well known, a direct comparison between their eﬃciency

in heterogeneous horizontal and vertical FL architectures is missing. Last but not least, the distribution of data held by

participants is a critical aspect to consider in FL, and there is a lack of work evaluating the robustness of FL models

trained with non-independent and identically distributed (non-IID) data.

To improve the previous challenges, this work presents the following main contributions:

•The design and implementation of three FL architectures, namely HoriChain, VertiChain, and VertiComb, one

for horizontal and two for vertical FL scenarios. HoriChain and VertiChain are inspired by a chain-based learning

protocol, while VertiComb follows a peer-to-peer network splitting strategy. The three architectures fully or

partially share the following characteristics: network architecture, training protocol, and dataset structure.

•The proposal of a distributed, decentralized, and privacy-preserving use case suitable for HFL and VFL that

uses non-IID data. In particular, the use case pretends to solve the problem of classifying handwritten digits in a

privacy-preserving way by splitting the MNIST dataset between seven participants. The three architectures are

executed using the same number of participants, number of adversaries, types of attacks, and implementations

of the attacks. Then, the performance of the three architectures is evaluated and compared. In conclusion, the

VertiChain architecture is less eﬀective than VertiComb and HoriChain.

•The evaluation of the HoriChain and VertiComb architectures robustness when trained in the previous scenario

and aﬀected by data and model poisoning attacks. The performed experiments show that diﬀerent conﬁgurations

of both attacks highly aﬀect the accuracy, F1-score, and learning time of both architectures. However, the

HoriChain architecture is more robust than the VertiComb when the attacks poison a reduced number of samples

and gradients.

The organization of this paper is as follows. First, related work dealing with FL and adversarial attacks are reviewed

in Section 2. Section 3details the FL architecture design. Section 4describes the use case, non-IID dataset splitting and

training pipeline in which the proposed architectures are tested. Section 5focuses on explaining the implementation

of adversarial attacks. The results and discussion of the performed experiments are evaluated in Section 6. Finally,

Section 7provides conclusions and draws future steps.

2. Related Work

This section reviews the state-of-the-art concerning FL architectures, adversarial attacks aﬀecting diﬀerent FL

scenarios, and works evaluating the robustness of FL models and architectures.

2.1. FL Scenarios and Architectures

In 2019, [12] deﬁned the scenarios of HFL, VFL, and FTL. The deﬁnitions use the symbols 𝑋to mean features,

𝑌for labels, 𝐼for the IDs of participants, and 𝐷for the local datasets. Then, an HFL scenario is characterized as

𝑋𝑖=𝑋𝑗, 𝑌𝑖=𝑌𝑗, 𝐼𝑖≠𝐼𝑗,∀𝐷𝑖, 𝐷𝑗, 𝑖 ≠𝑗. A VFL scenario can be identiﬁed as 𝑋𝑖≠𝑋𝑗, 𝑌𝑖≠𝑌𝑗, 𝐼𝑖=𝐼𝑗,∀𝐷𝑖, 𝐷𝑗, 𝑖 ≠

𝑗. Lastly, an FTL scenario has 𝑋𝑖≠𝑋𝑗, 𝑌𝑖≠𝑌𝑗, 𝐼𝑖≠𝐼𝑗,∀𝐷𝑖, 𝐷𝑗, 𝑖 ≠𝑗. The authors also distinguished FL from

distributed ML. Despite being very similar, in FL, users have autonomy and the central server cannot control their

participation in the training process. FL also has an emphasis on privacy protection, while distributed ML does not.

First Author et al.: Preprint submitted to Elsevier Page 2 of 15

Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning

The following year, [13] presented the client-server and peer-to-peer architectures for the HFL scenario. In the

client-server architecture, the server receives all model updates from participants (encrypted or in plain text, depending

on the scenario) and aggregates them. The peer-to-peer architecture is interesting because it eliminates the need for a

central coordinating point and its associated attack surface. In this approach, participants aggregating the models can

be randomly selected or follow a predeﬁned chain.

Concerning VFL, [14] introduced SplitNN, an architecture to train a shared model from participants holding diﬀer-

ent features and components (layers and nodes) of a neural network. Therefore, only the participant having a particular

model component knows its details. One participant trains locally its model components, and the outputs are passed

to another client, who holds the next component of the neural network. Finally, the participant controlling the ﬁnal

component in the neural network calculates the gradients and passes them back to the previous clients, who apply them

to their components.

2.2. Attacks in FL Scenarios

In [15], authors deﬁned honest-but-curious and malicious adversaries aﬀecting FL scenarios. Honest-but-curious

participants try to learn sensitive data and states of participants without deviating from the rules established by the FL

training protocol. In contrast, malicious participants try to destroy or corrupt the model without restrictions. Besides,

[16] focuses on malicious insider participants and poisoning attacks. Poisoning attacks can be categorized according

to diﬀerent criteria. One criterion deals with the attack objective. In this sense, random attacks aim to reduce the

accuracy of the trained FL model, whereas targeted attacks aim to inﬂuence the model to predict a given target label.

Another criterion is to target the data used to train the local model. In this direction, clean-label data poisoning attacks

assume that the adversary cannot change the label of any training data. Dirty-label attacks are when the adversary can

introduce any number of data samples. Finally, backdoor poisoning attacks modify individual features or a few data

samples to embed backdoors into the model. Overall, data poisoning attacks are less eﬀective in settings with fewer

participants.

In [17], the attack infers the participant’s training dataset from the gradients they share during training. The au-

thors develop a gradient-based feature reconstruction attack, in which the attacker receives the gradient update from a

participant and aims to steal their training set. The attacker iteratively reﬁnes the dummy image and label to approxi-

mate the real gradients. When they converge, the dummy training data converges to the real one with high conﬁdence.

[5] proposes a taxonomy with the diﬀerent attacks threatening an FL model. The taxonomy is organized into tables

with defenses and attacks. Attacks include the description and the source of the vulnerability that it exploits. On the

other hand, [18] creates a ﬂowchart-like visual representation of attacks and countermeasures. However, it only breaks

attacks into data privacy and model performance categories.

2.3. Robustness of HFL and VFL Architectures

Dealing with decentralized FL architectures for non-IID data aﬀected by heterogeneous adversarial attacks, [19]

explores backdoor attacks in a recommended system based on HFL. This work demonstrates the high impact of back-

doors attacks and that current defenses are not enough to solve the problem. Likewise, [20] proposes a ring-based

topology for FL focused on generative models. For security, the authors include a committee election method for

voting-based malicious node detection and a distributed model sharing scheme based on a decentralized ﬁle system.

Besides, there is a good number of decentralized FL works leveraging blockchain-based technologies for model sharing

and secure model tracking [21,22,23]. Regarding data privacy attacks, [24] proposes a framework for decentralized

FL but focused on privacy attack mitigation using secure cipher-based matrix multiplication. As it can be seen, there

are some works dealing with decentralized HFL and adversarial attacks.

The literature also has proposed solutions that evaluate the robustness of HFL using centralized model aggregation

approaches. In this sense, [25] trains several HFL models to detect cyberattacks aﬀecting IoT devices and considers

several conﬁgurations of label ﬂipping, data poisoning, and model cancelling attacks and model aggregation functions

acting as countermeasures. These functions provide a signiﬁcant improvement against malicious participants. Another

example is the proposed in [26], where HFL unsupervised and supervised models are trained to detect cyberattacks

aﬀecting spectrum sensors. Malicious participants implementing data and model poisoning attacks and four aggre-

gation functions acting as anti-adversarial mechanisms are considered to measure the model robustness. However,

despite the contributions of previous work, there is a lack of work focused on vertical FL that combines a decentralized

setting with the exploration of adversarial attacks. In this sense, the works present in the literature regarding attacks

in VFL focus on feature inference and privacy issues but do not consider model-focused attacks trying to degrade the

First Author et al.: Preprint submitted to Elsevier Page 3 of 15

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AnalyzingtheRobustnessofDecentralizedHorizontalandVerticalFederatedLearningArchitecturesinaNon-IIDScenarioPedroMiguelSánchezSáncheza,<,AlbertoHuertasCeldránb,EnriqueTomásMartínezBeltrána,DanielDemeterb,GérômeBovetc,GregorioMartínezPérezaandBurkhardStillerbaDepartmentofInformationandCommunicationsEng...

展开>> 收起<<

Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning Architectures in a Non-IID Scenario.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Analyzing the Robustness of Decentralized Horizontal and Vertical Federated Learning Architectures in a Non-IID Scenario

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: