Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts

2025-05-06 0 0 1.12MB 16 页 10玖币

侵权投诉

Decentralized Vision-Based Byzantine Agent

Detection in Multi-Robot Systems with IOTA

Smart Contracts

Sahar Salimpour, Farhad Keramat,

Jorge Pe˜na Queralta, and Tomi Westerlund

Turku Intelligent Embedded and Robotic Systems

University of Turku, Finland

{sahars, fakera, jopequ, tovewe}@utu.fi

https://tiers.utu.fi

Abstract. Multiple opportunities lie at the intersection of multi-robot

systems and distributed ledger technologies (DLTs). In this work, we in-

vestigate the potential of new DLT solutions such as IOTA, for detecting

anomalies and byzantine agents in multi-robot systems in a decentralized

manner. Traditional blockchain approaches are not applicable to real-

world networked and decentralized robotic systems where connectivity

conditions are not ideal. To address this, we leverage recent advances in

partition-tolerant and byzantine-tolerant collaborative decision-making

processes with IOTA smart contracts. We show how our work in vision-

based anomaly and change detection can be applied to detecting byzan-

tine agents within multiple robots operating in the same environment.

We show that IOTA smart contracts add a low computational overhead

while allowing to build trust within the multi-robot system. The pro-

posed approach eﬀectively enables byzantine robot detection based on

the comparison of images submitted by the diﬀerent robots and detec-

tion of anomalies and changes between them.

Keywords: Distributed ledger technologies ·Blockchain ·Deep learning

·Anomaly detection ·Change detection ·Multi-robot systems ·Com-

puter vision ·IOTA ·Smart Contracts ·Distributed Robotic Systems

1 Introduction

In recent years, byzantine agent detection has become an important aspect of dis-

tributed autonomous systems [6,10,29]. Indeed, with the growth and increasing

ubiquity of autonomous robots, security and safety issues for systems deployed

in the real world have attracted an ever-growing attention in both industrial

and academic areas [18,19]. As robotic systems are deployed in larger numbers,

single autonomous robots have been replaced by ﬂeets of multi-robot systems

that need to coordinate and collaborate. Many multi-robot applications, includ-

ing security monitoring, public safety [32], industrial applications [23,24], and

Internet of Things (IoT) systems [16], are at risk of being manipulated through

arXiv:2210.03441v1 [cs.RO] 7 Oct 2022

2 Sahar Salimpour et al.

1 2

3 4

(2) Image hashes and positions

are fed into the IOTA Tangle

(3) The smart contract triggers

image comparisons when

enough data is gathered

(4) Anomaly

detected

Trusted computing cloud

(1) Agents perceive the same

objects at different times

during their missions

(4) Data submitted by the robots is

compared in pairs to measure compliance

/ disparity between the robots.

Operational environment

Fig. 1: Conceptual illustration of the proposed vision-based byzantine agent detection

approach with IOTA smart contracts.

the injection of fabricated or noisy data, or the performance of a large system

might signiﬁcantly decrease because of a single malicious or byzantine actor.

Consequently, byzantine robots could potentially lead to a failure of the entire

multi-robot operation. Therefore, it is important to be able to detect and neu-

tralize the actions of byzantine agents, particularly if operating in environments

together with humans.

In multi-robot systems, vision-based perception often plays a major role in

use cases involving safety, surveillance, and environment monitoring. Vision-

based approaches to detect changes or anomalies in the environment can poten-

tially be used to also detect diﬀerences between sensing data gathered by diﬀerent

robots operating in a common environment. A majority of visual anomaly detec-

tion problems are focused on a speciﬁc class of images and attempt to identify

pixel-level anomalies in them. They mostly require training their deep learning

(DL)-based models using large amount of normal data [5,30,31]. However, the

detection of anomalies based on visual data in more general and potentially un-

known environments easily becomes a challenging task, especially in the context

of mobile robotic applications.

Novel approaches in the literature with potential to address the identiﬁca-

tion of byzantine robots in multi-robot systems are blockchain-based solutions

through smart contracts. Blockchain technology was originally developed for the

purposes of ﬁnancial transactions [17], but it has also been utilized as a dis-

tributed computing framework for applications in general, e.g., within the Inter-

net of Things (IoT) domain, as well as in multi-robot systems. A distributed sys-

tem integrating blockchain technology is a priori capable of delivering a trusted

and decentralized system between independent and untrusted agents. In the case

Decentralized Vision-Based Byzantine Agent Detection with IOTA 3

of autonomous robots, this allows for decentralized collaborative decision making

without the need for a third-party central organization. By doing so, a consis-

tent global state makes the whole system resilient and fault-tolerant against

byzantine robots.

IOTA smart contracts, designed for IoT devices, are one of the promising

distributed ledger technology (DLT) solutions that can be used in multi-robot

systems. In our previous work [10], we have presented a general partition-tolerant

and byzantine-tolerant framework built on top of IOTA smart contracts and in-

tegrated to ROS 2. By leveraging this framework, all non-byzantine robots could

reach a consensus about which robot is byzantine in a decentralized manner.

Blockchains or other distributed ledger technologies (DLTs) have potential

to be an innovative solution to vision applications. However, to the best of our

knowledge, no studies have been conducted on this topic within the context of

multi-robot systems. In this study, we present a framework to detect byzan-

tine robot(s) in a secure network and operating in a common environment by

analyzing the RGB images which are captured by each robot. In the proposed

method we use our previous study [25], presenting a general framework to detect

anomalies and changes between images, to compare in pairs images gathered by

diﬀerent robots. In this case, an anomaly could be something that has been

moved or removed from the environment or something that does not belong

there, as well as potentially altered or fabricated data.

This paper therefore integrates a vision-based approach for anomaly and

change detection in autonomous inspection robots together with IOTA smart

contracts, as illustrated by Fig. 1. The result is a decentralized solution to

anomaly detection that can be applied to byzantine agent detection within multi-

robot systems. The blockchain serves as a tool for storing agent locations and

image hashes, while smart contracts calculate where and when to perform the

anomaly and change detection once enough data has been acquired. The DL

model itself runs on a trusted server, owing to the impossibility of integrating

such complex computation (deep neural networks) within a smart contract, and

therefore limiting the decentralization of the solution. However, this is a ﬁrst step

towards a fully distributed implementation where multiple nodes will be able to

validate the output of the DL models. In summary, the main contributions of

this work are the following:

i) The design and implementation of a blockchain-based approach to byzantine

agent detection using IOTA smart contract and vision sensors;

ii) the extension of our previous work in anomaly and change detection for

autonomous inspection robots to comparing data from multiple robots op-

erating in the same environment towards byzantine agent detection; and

iii) the integration of the DL models with IOTA smart contracts that trigger

data comparisons after tracking the position of robots and the location of

gathered data.

The rest of the manuscript is structured as follows. Section II discusses re-

lated research on blockchain technology in robotic systems, and the problem of

4 Sahar Salimpour et al.

anomaly detection in multi-robot systems. A general introduction on the back-

ground is given in section III, and a description of our methodological approach

is provided in section IV. Section V presents the results, and Section VI sum-

marizes the work and points to future directions.

2 Related Work

Generally, byzantine and fault detection in robotics can be divided into two

major groups: self-monitoring and group-monitoring anomaly detection. Sev-

eral studies proposed the self-monitoring approach, in which each robot detects

anomalies independently. A framework for detecting mechanical faults and sensor

faults in wheeled robots was presented in [33]. Tingting et al [3]. proposed an un-

supervised anomaly detection model using a sliding-window convolutional varia-

tional autoencoder in terms of time series eﬀect. However, swarm-level anomaly

detection methods analyze the collaboration between robots and the data col-

lected from the entire swarm [15].

Many studies have addressed byzantine robot detection in multi-robot mis-

sions using blockchain technology. In [28] a blockchain-based approach was ex-

plored for swarm robotics systems with byzantine robots. The authors utilized

Ethereum-based decentralized smart contracts to detect and remove the byzen-

tine swarm members. Their approach was evaluated using a collective decision-

making scenario in which robots must agree on the most frequent tile color in

an environment. In another work [6], a blockchain was used as a secure com-

munication tool in Byzantine Follow The Leader (BFTL) missions. Through

their approach, leader robots guide follower robots to speciﬁc destinations under

the threat of Byzantine robots misdirecting them. In addition, some research

conducted to implement blockchain protocols into secure communication multi-

agent systems with unmanned aerial vehicles [1,9].

The immutability of blockchain makes it a secure solution for detecting

anomalous behaviors and attacks in various systems with chains of informa-

tion blocks, such as industrial control systems [8], electricity consumption [11],

and health systems [2]. The authors in [12] implemented smart contracts to store

robot information and compute them to detect anomalies, which were simulated

internal failure, in machinery and register them in the blockchain. Golomb et

al. [7] introduced a collaborative anomaly detection model for a large network of

IoT devices by leveraging blockchain technology in conjunction with extensible

Markov model.

A number of recent studies have focused on detecting visual anomalies within

speciﬁc image classes, such as railway images [31], road datasets [26], and in-

dustrial production images [22]. In a recent visual-based blockchain task [21],

the technology of blockchain was used to provide decentralized communication

between robots so they could ﬁnd their way back home using common visual

landmarks. In [13], external parties, Oracle, analyze captured images to deter-

mine how many balls need to be picked by UR3 arm. With this information,

smart contracts can securely control robots, ensuring that no one can change

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DecentralizedVision-BasedByzantineAgentDetectioninMulti-RobotSystemswithIOTASmartContractsSaharSalimpour,FarhadKeramat,JorgePe~naQueralta,andTomiWesterlundTurkuIntelligentEmbeddedandRoboticSystemsUniversityofTurku,Finlandfsahars,fakera,jopequ,toveweg@utu.fihttps://tiers.utu.fiAbstract.Multipleopport...

展开>> 收起<<

Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts.pdf

共16页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: