Assuring Safety of Vision-Based Swarm Formation Control Chiao Hsieh1 Yubin Koh1 Yangge Li1and Sayan Mitra1 Abstract Vision-based formation control systems are at-

2025-04-27 0 0 1.69MB 8 页 10玖币

侵权投诉

Assuring Safety of Vision-Based Swarm Formation Control

Chiao Hsieh1, Yubin Koh1, Yangge Li1and Sayan Mitra1

Abstract— Vision-based formation control systems are at-

tractive because they can use inexpensive sensors and can

work in GPS-denied environments. The safety assurance for

such systems is challenging: the vision component’s accuracy

depends on the environment in complicated ways, these errors

propagate through the system and lead to incorrect control

action, and there exists no formal speciﬁcation for end-to-end

reasoning. We address this problem and propose a technique

for safety assurance of vision-based formation control: First, we

propose a scheme for constructing quantizers that are consistent

with vision-based perception. Next, we show how the conver-

gence analysis of a standard quantized consensus algorithm

can be adapted for the constructed quantizers. We use the

recently deﬁned notion of perception contracts to create error

bounds on the actual vision-based perception pipeline using

sampled data from different ground truth states, environments,

and weather conditions. Speciﬁcally, we use a quantizer in

logarithmic polar coordinates, and we show that this quantizer

is sutiable for the constructed perception contracts for the

vision-based position estimation, where the error worsens with

respect to the absolute distance between agents. We build our

formation control algorithm with this nonuniform quantizer,

and we prove its convergence employing an existing result for

quantized consensus.

I. INTRODUCTION

Distributed consensus, ﬂocking, and formation control

have been studied extensively, including in scenarios where

the participating agents only have partial state information

(see, for example [1]–[4]). With the advent of deep learn-

ing and powerful computer vision algorithms, it is now

feasible for agents to use vision-based state estimation for

formation control (See Figure 1). Such systems can be

attractive because they do not require expensive sensors and

localization systems, and also can be used in GPS-denied

environments [5]–[7]. However, deep learning and vision

algorithms are well-known to be fragile, which can break the

correctness and safety of the end-to-end formation control

system. Further, it is difﬁcult to specify the correctness of

a vision-based state estimator, which gets in the way of

modular design and testing of the overall formation control

system [8]. In this paper, we address these challenges and

present the ﬁrst end-to-end formal analysis of a vision-based

formation control system.

We present analyses for both convergence and safety

assurance of a vision-based swarm formation control system.

The computer vision pipeline (See Figure 2) uses feature

detection, feature matching, and geometry to estimate the

relative position of the participating drones. The estimated

relative poses are then used by a consensus-based formation

1The authors are with Coordinated Science Laboratory, University of Illi-

nois Urbana-Champaign, Champaign, IL, USA {chsieh16, yubink2,

li213, mitras}@illinois.edu

Fig. 1: Vision-based drone formation using downward facing

camera images in AirSim.

control algorithm. There are two key challenges in analyzing

the system: (1) The perception errors impact the behavior

of neighboring agents and propagates through the entire

swarm. (2) The magnitude of the perception error is highly

nonuniform, and depends on the ground truth values of the

relative position between neighboring agents. In general,

perception errors can get worse as the system approaches the

equilibrium (desired formation), and thus, make stabilization

difﬁcult. Environmental variations (e.g., lighting, fog) are

other factors that can make the vision-based system unstable.

In addressing the problem, our idea is to view the vision-

based formation control system as a quantized consensus

protocol [9]. We start with the assumption that the impact

of the state estimation errors arising from the vision pipeline

can be encapsulated as quantization errors in a non-uniform

quantization scheme. That is, the quantization step size

can vary non-uniformly with respect to the state so that

the quantization errors can overapproximate state dependent

perception errors. To discharge this assumption, our analysis

has to meet two requirements. First, we have to propose

aspeciﬁc quantization scheme under which the formation

control system is indeed guaranteed convergence. For this,

we develop a quantized formation controller (Equation (1))

and a logarithmic polar quantizer (Equation (2) and (3)), and

we show in Theorem 1 that indeed the resulting quantized

formation control protocol converges, using sufﬁcient condi-

tions from [9]. Secondly, we have to show that a quantizer

instantiated from the quantization scheme matches the error

characteristics of the vision pipeline. For this part, we utilize

the recently developed idea of perception contracts [10],

[11]. A perception contract (PC) for a vision-based state

estimator bounds the estimation error as a function of the

ground truth state. Earlier in [11], PCs have been used to

establish the safety of vision-based lane keeping systems.

For formation control, however, the PCs are dramatically

different because the error has a highly non-uniform de-

arXiv:2210.00982v2 [cs.MA] 27 Sep 2023

Fig. 2: Architecture of an agent in the vision-based formation

control systems.

pendency on the state; as the drones get closer, the error

drops. Through data-driven construction of the logarithmic

PC, we show that the vision pipeline indeed matches the PC

with high probability in Section IV, and we further adapt to

environmental variations by inferring quantization step sizes

for different environments.

In summary, our contributions are as follows: (1) An

approach to construct a quantizer as the perception contract

of the vision component. (2) Empirical analysis of the impact

of environmental variations on the perception contract with

the photorealistic AirSim simulator [12]. (3) Theoretical

analysis of the overall formation control system using the

constructed quantizer, which gives the bounds on the con-

vergence time. Our code of the vision pipeline, simulation

script, and analysis tool are publicly available1.

Related Works: Two parallel threads of research have

recently addressed formal end-to-end analysis of vision-

based autonomous systems. The works in [13]–[15] approach

the problem using discrete state space and stochastic models.

Our previous work [10], [11] develops the idea of perception

contracts using the language of continuous state space mod-

els. Thus far all the applications studies in these threads are

related to lane following by a single agent, which is quite

different from distributed formation control.

VerifAI [16] uses techniques like fuzz testing and simu-

lation to falsify the system speciﬁcations. Katz et al. [17]

trains generative adversarial networks (GANs) to produce

a network to simplify the image-based NN. NNLander-

VeriF [18] veriﬁes NN perception along with NN controllers

for an autonomous landing system. In contrast, our current

work aims to provide safety analyses for a formation control

system with vision-based perception, and we apply the

analysis on convergence to quantized consensus [9] for safe

separation and formation.

Paper Organization: In Section II, we introduce the

formation control system with the vision-based perception

and review the quantized formation controller. In Section III,

we show the convergence under perception error using our

main theory of quantized consensus. In Section IV, we

describe the quantization for perception error bounds via

sampling from vision-based pose estimation with AirSim

1Repository: https://gitlab.engr.illinois.edu/aap/airsim-vision-formation

Fig. 3: Feature matching on an image pair from AirSim.

simulation. We then conclude in Section V.

II. VISION-BASED FORMATION CONTROL

We will study a distributed formation control system with

N+1 identical aerial vehicles or agents with a leader agent

0 as shown in Figure 1. The target formation is speciﬁed

in terms of relative positions between agents. Each agent i

has a downward facing camera, and it uses images from its

own camera and its predecessor i−1’s camera to periodically

estimate the relative position of iwith respect to i−1. Based

on the estimated relative positions to its neighbor, agent i

then updates it own position by setting a velocity, to try and

achieve the target formation.

Before describing the vision and control modules in more

detail, we introduce some notations. First, we describe the

neighborhood relation between agents by an undirected con-

nected graph G= (V,E), where V={0,1,...,N}. Second,

we only consider planar formations for simplicity though the

agents are in 3-dimensional space. Thus, the position of agent

iin the world frame is represented by a vector qi∈R2. The

state of the overall system is a sequence q = (q0,q1,...,qN).

The distributed formation control system evolves with a goal

of reaching a target formation in a set Eq∗that is speciﬁed

by a vector q∗= (q∗

0,q∗

1,...,q∗

N)as

Eq∗={q| ∀i∈ {1,...,N},qi−qi−1=q∗

i−q∗

i−1}.

That is, Eq∗is the set of all states that form q∗up to

translations. We also specify a safe set that the where the

distance between no two agents is too close or too far:

Sq={q| ∀i∈ {1,...,N},dmin <∥qi−qi−1∥<dmax}

where 0 <dmin <dmax deﬁnes the range of safe distances.

A. Vision-Based Relative Pose Estimation

We now discuss the components of an agent i(Figure 2).

Agent i’s downward-facing camera speriodically generates

an image of the ground mi, which depends on its state qi

and other environmental factors like background scenery,

lighting, fog, etc. The neighboring agent i−1 generates

another image mi−1of the ground and shares this with

agent iover the communication channel. We assume the

whole system runs synchronously in lock-step, i.e., both

neighboring drones will capture the image at the same time

and there’s no communication delay between drones while

sharing images. The vision-based pose estimation algorithm

htakes a pair of images, mi−1and mi, as an input and

produces the estimated relative position ˆyito estimate the

relative position of agent iwith respect to agent i−1, i.e.,

qi−qi−1. The estimation algorithm in general follows these

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AssuringSafetyofVision-BasedSwarmFormationControlChiaoHsieh1,YubinKoh1,YanggeLi1andSayanMitra1Abstract—Vision-basedformationcontrolsystemsareat-tractivebecausetheycanuseinexpensivesensorsandcanworkinGPS-deniedenvironments.Thesafetyassuranceforsuchsystemsischallenging:thevisioncomponent’saccuracydepe...

展开>> 收起<<

Assuring Safety of Vision-Based Swarm Formation Control Chiao Hsieh1 Yubin Koh1 Yangge Li1and Sayan Mitra1 Abstract Vision-based formation control systems are at-.pdf

共8页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Assuring Safety of Vision-Based Swarm Formation Control Chiao Hsieh1 Yubin Koh1 Yangge Li1and Sayan Mitra1 Abstract Vision-based formation control systems are at-

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: