ECCV 2022 Workshop on Adversarial Robustness in the Real World Tel Aviv Israel. Attacking Motion Estimation with Adversarial Snow Jenny Schmalfuss Lukas Mehl Andr es Bruhn

2025-05-03 0 0 9.62MB 5 页 10玖币

侵权投诉

ECCV 2022 Workshop on Adversarial Robustness in the Real World, Tel Aviv, Israel.

Attacking Motion Estimation with Adversarial Snow

Jenny Schmalfuss Lukas Mehl Andr´

es Bruhn

Institute for Visualization and Interactive Systems, University of Stuttgart, Germany

firstname.lastname@vis.uni-stuttgart.de

Abstract

Current adversarial attacks for motion estimation (opti-

cal ﬂow) optimize small per-pixel perturbations, which are

unlikely to appear in the real world. In contrast, we ex-

ploit a real-world weather phenomenon for a novel attack

with adversarially optimized snow. At the core of our at-

tack is a differentiable renderer that consistently integrates

photorealistic snowﬂakes with realistic motion into the 3D

scene. Through optimization we obtain adversarial snow

that signiﬁcantly impacts the optical ﬂow while being indis-

tinguishable from ordinary snow. Surprisingly, the impact

of our novel attack is largest on methods that previously

showed a high robustness to small Lpperturbations.

1. Introduction

Adversarial attacks, which are a severe threat to neural

networks, have recently been introduced in the context of

optical ﬂow. There, the goal is to compute the pixel-wise

2D motion fbetween two consecutive frames of an image

sequence at times tand t+1. Current attacks [11,14] modify

these two frames in the 2D space and consequently ignore

the actual 3D geometry of the scene and the objects moving

within. Moreover, when modifying pixels, they do not im-

pose visual constraints, yielding attacked images that lack

naturalism. Therefore, the conclusions drawn from robust-

ness analyses with these attacks might not necessarily re-

ﬂect the robustness of optical ﬂow methods in the real world

– where perturbations are more likely to appear in the form

of weather phenomena.

This work aims to answer the question whether a natu-

rally occurring weather effect like snow can be manipulated

to serve as an adversarial sample for motion estimation. To

this end, we propose an adversarial attack that augments

images with falling snowﬂakes featuring a high degree of

realism: We create snowﬂakes with a view-consistent 3D

motion over time, insert them into the 3D scene in a depth-

aware manner, and ensure photo-realism through visual ef-

fects (see Fig. 1). This enables us to generate adversari-

ally manipulated snow that signiﬁcantly deteriorates optical

Original

Frame tFrame t+1 Optical ﬂow

Random snow

Adversarial snow

Figure 1. Snow attack with 3000 snowﬂakes that are ﬁrst placed

randomly in the 3D scene (Random snow) and then optimized (Ad-

versarial snow) to perturb optical ﬂow estimation with GMA [6].

ﬂow predictions, while still satisfying the spatio-temporal

and visual constraints of naturalistic snow. We consider

snow as representative weather effect where single particles

move independent of the remaining scene content, but note

that the proposed attack procedure could also be used to

model rain or sleet.

Related work. Current optical ﬂow methods based on neu-

ral networks [5,6,10] were recently shown to be suscep-

tible to adversarially modiﬁed input images, which alter

the resulting attacked ﬂow ˇ

fto resemble a speciﬁed tar-

get ﬂow fT. The few existing adversarial attacks on optical

ﬂow methods generate either perturbations with small Lp

norms [14,15] or adversarial patches [11], while adversarial

weather attacks are completely unexplored. In contrast, ad-

versarial perturbations that imitate snow effects have been

investigated in the context of classiﬁcation [7,8] or hu-

man pose estimation [16]. However, for these applications,

weather attacks [7,8] or snow augmentations [4,9] only

have to be applied to single images rather than sequences.

For optical ﬂow estimation, a realistic motion of the weather

effect over multiple frames and camera perspectives is

required, imposing certain geometric constraints in time,

which prevents the direct application of existing single-

image adversarial weather generation schemes. Also, learn-

ing models for variations in images from data rather than

modeling them explicitly has been explored for adversar-

arXiv:2210.11242v1 [cs.CV] 20 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ECCV2022WorkshoponAdversarialRobustnessintheRealWorld,TelAviv,Israel.AttackingMotionEstimationwithAdversarialSnowJennySchmalfussLukasMehlAndr´esBruhnInstituteforVisualizationandInteractiveSystems,UniversityofStuttgart,Germanyfirstname.lastname@vis.uni-stuttgart.deAbstractCurrentadversarialattacksfor...

展开>> 收起<<

ECCV 2022 Workshop on Adversarial Robustness in the Real World Tel Aviv Israel. Attacking Motion Estimation with Adversarial Snow Jenny Schmalfuss Lukas Mehl Andr es Bruhn.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

ECCV 2022 Workshop on Adversarial Robustness in the Real World Tel Aviv Israel. Attacking Motion Estimation with Adversarial Snow Jenny Schmalfuss Lukas Mehl Andr es Bruhn

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: