1 Common Corruption Robustness of Point Cloud Detectors Benchmark and Enhancement

2025-04-27 0 0 3.72MB 16 页 10玖币

侵权投诉

Common Corruption Robustness of Point Cloud

Detectors: Benchmark and Enhancement

Shuangzhi Li, Zhijie Wang, Felix Juefei-Xu, Qing Guo*, Xingyu Li, and Lei Ma

Abstract—Object detection through LiDAR-based point cloud

has recently been important in autonomous driving. Although

achieving high accuracy on public benchmarks, the state-of-the-

art detectors may still go wrong and cause a heavy loss due

to the widespread corruptions in the real world like rain, snow,

sensor noise, etc. Nevertheless, there is a lack of a large-scale

dataset covering diverse scenes and realistic corruption types

with different severities to develop practical and robust point

cloud detectors, which is challenging due to the heavy collection

costs. To alleviate the challenge and start the ﬁrst step for robust

point cloud detection, we propose the physical-aware simulation

methods to generate degraded point clouds under different real-

world common corruptions. Then, for the ﬁrst attempt, we

construct a benchmark based on the physical-aware common

corruptions for point cloud detectors, which contains a total of

1,122,150 examples covering 7,481 scenes, 25 common corruption

types, and 6 severities. With such a novel benchmark, we conduct

extensive empirical studies on 8 state-of-the-art detectors that

contain 6 different detection frameworks. Thus we get several

insight observations revealing the vulnerabilities of the detectors

and indicating the enhancement directions. Moreover, we fur-

ther study the effectiveness of existing robustness enhancement

methods based on data augmentation and data denoising. The

benchmark can potentially be a new platform for evaluating point

cloud detectors, opening a door for developing novel robustness

enhancement methods.

Index Terms—Point cloud, Object Detection, Benchmark, Ro-

bustness

I. INTRODUCTION

Bject detection via LiDAR-based point cloud [

], [

], as

a crucial task in 3D computer vision, has been widely

used in applications like autonomous driving [

]. Recently,

the data-driven methods (i.e., deep neural networks) have

signiﬁcantly improved the performance of 3D point cloud

detectors [

], [

] on various public benchmarks, e.g., KITTI

[

], NuScenes [

], and Waymo [

]. However, the scenarios

covered by these public benchmarks are usually limited. For

instance, there is a lack of natural fog effects in these datasets,

while fog could affect the reﬂection of laser beams and

corrupt point cloud data with false reﬂections by droplets [

[

]. Apart from the external scenarios, the internal noise of

sensors can also increase the deviation and variance of ranging

*Qing Guo is the corresponding author.

Shuangzhi Li, Zhijie Wang, Xingyu Li, and Lei Ma are with the University

of Alberta, AB, Canada. Zhijie Wang and Lei Ma are also with the Alberta

Machine Intelligence Institute, AB, Canada. Lei Ma is also with Kyushu

University, Japan. (e-mail: {shuangzh, zhijie.wang, xingyu}@ualberta.ca,

ma.lei@acm.org)

Qing Guo is with the Nanyang Technological University, Singapore. (e-mail:

tsingqguo@ieee.org)

Felix Juefei-Xu is with New York University, New York, NY 10012, USA.

(e-mail: juefei.xu@nyu.edu)

measurements [

] and result in corrupted data and detector

performance degradation. Given that LiDAR-based point cloud

detection is usually used in safety-critical applications (e.g.,

autonomous driving) and these external and internal corruptions

could potentially affect detectors’ robustness [

], [

it is critical to comprehensively evaluate an object detector

under those corruptions before deploying it in real-world

environments.

There are some works constructing datasets while consider-

ing extreme weather like CADC [

], Boreas [

], SeeThrough-

Fog (STF) [

]. Nevertheless, the constructed datasets only

consider limited situations in the real world due to the heavy

collection costs, which are far from a comprehensive evaluation.

For instance, Boreas only covers 4 rainy scenes and 5 snowy

scenes. STF only contains foggy point clouds at severity levels

of “dense” and “light”. Hence, there is an increasing demand

for extending existing benchmarks to conduct a comprehensive

evaluation through covering diverse corruptions in the real

world. A straightforward way is to synthesize the corrupted

point clouds given the success of similar solutions in the image-

based tasks [

], [

] and 3D object recognition [

], [

However, there is no accessible dataset for the robustness

evaluation of point cloud detectors. Note that, the robustness

datasets (e.g., Modelnet40-C [

]) for 3D object recognition

cannot be used to evaluate the point cloud detectors, directly:

(1) the example in the recognition dataset only contains the

points of an object and cannot be adopted for object detection

task that aims to localize and classify objects in 3D scene.

(2) The latest Modelnet-C [

] and Modelnet40-C [

]) only

consider 7 corruptions and 15 corruptions, respectively, which

is still limited for a comprehensive evaluation in safety-critical

environments such as autonomous driving.

The main challenge for building a dataset for the robustness

evaluation of point cloud detection stems from the huge amount

of diverse corruption types with different physical imaging

principles. For example, ﬂawed sensors and different object

characteristics could lead to noise-like corruptions and affect

spherical and Cartesian coordinates of points, respectively.

Different weathers like rain and fog might lead to false

reﬂections. These corruptions have different imaging principles

and need careful designs of the respective simulation methods.

In this work, for the ﬁrst attempt, we construct a benchmark

to evaluate the robustness of point cloud object detectors based

on LiDAR under diverse common corruptions and discuss

the effectiveness of existing robustness enhancement methods.

Regarding the benchmark construction, we ﬁrst design physical-

aware simulation methods for

corruptions according to

their physical models, respectively. Then, we borrow 7,481

arXiv:2210.05896v1 [cs.CV] 12 Oct 2022

TABLE I: Summary of datasets used for LiDAR-based point cloud object detection

Dataset Year Real/Simulated Frames BBoxes Classes Corruptions Corruption

Severities

Robustness

Metric

KITTI [6] 2012 real 15K 200K 8 cutout, noise 2 -

NuScenes [7] 2019 real 400K 1.4M 23 rain, sun, clouds, cutout, various vehicle types, noise 2 -

Waymo [8] 2019 real 200K 12M 4 rain, fog, cutout, dust, various vehicle types, noise 2 -

Boreas [15] 2022 real 7.1K 320K 4 snow, rain, sun, clouds, cutout, noise 2 -

STF [10] 2020 real 13.5K 100K 4 fog, rain, snow, cutout, noise 3 -

CADC [14] 2020 real 7K 334K 10 snow, bright light, cutout, noise 5 -

ModelNet40-C [11] 2022 real+simulated 185K - 40

occlusion, LiDAR, local_density_inc/dec, cutout,

uniform, Gaussian, impulse, upsampling, background,

rotation, shear, FFD, RBF, inv_RBF

ModelNet-C [18] 2022 real+simulated 185K - 40 scale, rotate, jitter, drop_global/local, add_global/local 6X

Argoverse [19] 2019 real 468K 993K 15 rain, cutout, dust, noise 2 -

Lyft Level 5 [20] 2020 real 30K 1.3M 9 rain, cutout, noise 2 -

Ours 2022 real+simulated 1.1M 15M 8

Scene: rain, snow, fog, uniform_rad, gaussian_rad,

impulse_rad, upsample, background, cutout, beam_del,

local_dec/inc, layer_del; Object: uniform, gaussian,

impulse, upsample, cutout, local_dec/inc, shear, scale,

rotation, FFD, translation

raw 3D scenes (i.e., clean point clouds) from [

] and build

large-scale corrupted datasets by adding

corruptions with

different severity levels to each clean point cloud. Finally, we

obtain a total of 1,122,150 examples covering 7,481 scenes,

25 common corruption types, and 6 severity levels. Compared

with real-world data benchmark (see Table I), the proposed

benchmark synthesized more examples for benchmarking

robustness. Compared with other synthesized benchmark (see

Table I), our benchmark provides more types of corruption

patterns to speciﬁcally support benchmarking object detection.

Note that, we conduct extensive experiments to quantitatively

validate the effectiveness of simulation methods by evaluating

the naturalness of synthesized data.

With such a novel benchmark, we investigate the robustness

of current point cloud detectors by conducting extensive

empirical studies on 8 existing detectors, covering 3 different

representations and 2 different proposal architectures. In

particular, we study the following four research questions to

identify the challenges and potential opportunities for building

robust point cloud detectors:

•How do the common corruption patterns affect the point

cloud detector’s performance?

Given overall common

corruptions, an accuracy drop of

11.01%

(on average) on all

detectors anticipates a noticeable accuracy drop of detectors

against diverse corruption patterns.

•How does the design of a point cloud detector affect its

robustness against corruption patterns?

Compared with

two-stage detectors, one-stage detectors perform more robust

against a majority of corruptions. Compared with point-based

detectors, voxel-involving detectors perform more robust

against the most of corruptions.

•What kind of detection bugs exist in point cloud detec-

tors against common corruption patterns?

Followed by

the decrease in the rate of true detection, common corruptions

widely trigger a number of false detections on all point cloud

detectors.

•How do the robustness enhancement techniques im-

prove point cloud detectors against common corruption

patterns?

Even with the help of data augmentation and

denoising, common corruptions still cause a severe accuracy

drop of over 10% on detection.

In summary, this work makes the following contributions:

•

We design physical-aware simulation methods covering

25 common corruptions related to natural weather, noise

disturbance, density change, and object transformations at

the object and scene level.

•

We create the ﬁrst robustness benchmark of point cloud

detection against common corruptions.

•

Based on the benchmark, we conduct extensive empirical

studies to evaluate the robustness of 8 existing detectors

to reveal the vulnerabilities of the detectors under common

corruptions.

•

We study the existing data augmentation (DA) method and

denoising method’s performance on robustness enhancement

for point cloud detection and further discuss their limitations.

II. RELATED WORK

A. LiDAR Perception

LiDAR perception is sensitive to both internal and external

factors that could result in different corruptions. Adversarial

weather [

] (e.g., snow, rain, and fog) can dim or even block

transmissions of lasers by dense liquid or solid droplets. Regard-

ing noise characteristics of point clouds, strong illumination

[

] affects the signal transmission by lowering Signal-to-Noise

Ratio (SNR), increasing the noise level of LiDAR ranging [

Besides, the intrinsically inaccurately ranging and the sensor

vibration [

], [

] potentially trigger noisy observations during

LiDAR scanning. Environmental ﬂoating particles (e.g., dust

[

]) could perturb point cloud with the background noise.

Density distribution of LiDAR-based point clouds can also

easily affect autonomous driving. For instance, common object-

object occlusions block LiDAR scanning on objects in the

scene [

]. Besides, the dark-color cover and rough surface

TABLE II: Taxonomy of collected common corruption patterns

Scene-level Object-level

Corruption

Category Corruption Potential Reasons Corruption

Category Corruption Potential Reasons

Weather

rain

Environment: natural weather [9]; Noise

uniform

Object surface: coarse surface [21]

and dark-color cover [21];

snow gaussian

fog impulse

Noise

uniform_rad Environment: strong illumination [22];

Sensor: low ranging accuracy [23] and

sensor vibration [24], [25];

upsample

gaussian_rad

Density

cutout Object surface: object or self-

occlusions [13], dark-color cover [21]

and transparent components;

impulse_rad local_dec

upsample local_inc

background Environment: ﬂoating particles [26];

Transformation

translation Object: different locations and

heading directions [27];

Density

cutout

Sensor: different scanning layers, object

occlusion [13], and randomly laser beam

[13] or layer (rotary laser) malfunction;

rotation

local_dec shear Object deformation: bending or

moving pedestrians [28], different

styles of vehicles [29].

local_inc FFD

beam_del scale

layer_del

[

] could affect LiDAR’s reﬂection and thus reduce local point

density when sensing such objects. Moreover, the malfunction

of (ﬁxed or rotary) lasers [

] globally loses points or layers

of points in point clouds. For 3D tasks, various shapes [

[

], locations and poses [

] of objects can also inﬂuence the

context perception in the scene.

Apart from these natural corruptions, LiDAR perception is

also sensitive to adversarial attack. Adversarial attacks [

] pose

signiﬁcant security issues and vulnerability on 3D point cloud

tasks (e.g., classiﬁcation [

], detection [

], and segmentation

[33]).

B. Point Cloud Detectors

Based on the different representations acquired from point

clouds, point cloud detectors can be categorized into

2D-view-

based

detectors (e.g., VeloFCN [

] and PIXOR [

]),

voxel-

based detectors

(e.g., SECOND [

] and VoTr [

]),

point-

based

detectors (e.g., PointRCNN [

] and 3D-SSD [

]),

and

point-voxel-based

detectors (e.g., PVRCNN [

] and SA-

SSD [

]). On the other hand, based on the different proposal

architectures, point cloud detectors can also be divided into

one-stage

detectors (e.g., 3D-SSD [

] and SA-SSD [

]) and

two-stage

detectors (e.g., PointRCNN [

] and PVRCNN [

]).

In this paper, we select 8 representative methods covering all

these categories.

C. Robustness Benchmarks against Common Corruptions

Several attempts have been made to benchmark robustness

for different data domains. Based on ImageNet [

], ImageNet-

C simulates real-world corruptions to test image classiﬁers’

robustness. ObjectNet [

] illustrates the performance degrada-

tion of 2D recognition models considering object backgrounds,

rotations, and imaging viewpoints. Inspired by 2D works,

several benchmarks were built for 3D tasks. Modelnet40-C

[

] corrupts ModelNet40 [

] with 15 simulated common

corruptions affecting point clouds’ noise, density, and transfor-

mations, to evaluate the robustness of point cloud recognition.

Targeting 7 fundamental corruptions (i.e., “Jitter”, “Drop

Global/Local”, “Add Global/Local”, “Scale”, and “Rotate”),

ModelNet-C reveals the vulnerability of different components

of 9 existing point cloud classiﬁers. Regarding point cloud

detection, NuScenes, Waymo, and STF collect LiDAR scans

under adversarial rainy, snowy, and foggy conditions, where

the accuracy of 3D detectors is tested [

], [

]. However,

to the best of our knowledge, a lack of benchmark of point

cloud detection’s robustness comprehensively against various

common corruptions is still remaining.

D. Robustness Enhancement for Point Cloud Detection

Recently, improving the robustness of point cloud detection

has also received signiﬁcant concerns. Zhang et al. propose

PointCutMix [

] as a single way to generate new training

data by replacing the points in one sample with their optimal

assigned pairs in another sample. Lee et al. [

] propose a rigid

subset mix (RSMix) augmentation to get a virtual mixed sample

by replacing part of the sample with shape-preserved subsets

from another sample. Speciﬁcally for 3D object detection, there

are several ways to improve detectors’ robustness. Choi et al.

[

] propose a part-aware data augmentation that stochastically

augments the partitions of objects by 5 basic augmentation

methods. LiDAR-Aug [

] presents a rendering-based LiDAR

augmentation framework to improve the robustness of 3D

object detectors. LiDAR light scattering augmentation [

] and

LiDAR fog stimulation [

] utilize physics-based simulators

to generate data corrupted by fog/snow/rain and then augment

object detectors. Self-supervised pre-training [

], [

] can

also endow the model with resistance to augmentation-related

transformations. Besides, denoising methods [

], [

]

can remove the outliers in point clouds and thus potentially

improve detectors’ robustness. Regarding module design, there

are also some detectors specialized for resisting corruptions,

e.g. BtcDet [

] with the occupancy estimator for estimating

occluded regions and Centerpoint [

] with key-point detector

for a ﬂexible orientation regression. In this paper, we evaluate

part-aware data augmentation and K-nearest-neighbors-based

ﬁltering methods for improving point cloud detectors against

diverse common corruption patterns.

III. BACKGROUND

A. Point Cloud Detection

Point clouds detectors aim to detect objects of interest

in point clouds in the format of bounding boxes (BBoxes).

Suppose a frame of point cloud data

is a set of point

p= [xp, yp, zp, rp]

, where

(xp, yp, zp)

denotes its 3D location

and

denotes reﬂective intensity. Thus we can formulate the

point cloud detection as:

Det(P) = {bi}N

bi= [xi, yi, zi, wi, hi, li, θi, ci, si](1)

where

Det(·)

represents the detector;

is the number of

detected BBoxes in

;

denotes

ith

detected BBox in

, where

i= 1,2,· · · , N

;

(xi, yi, zi)

is the Cartesian coordinate of the

center of

(wi, hi, li)

is its dimensions,

θi

is its heading

angle,

is its classiﬁcation label, and

is its prediction

conﬁdence score.

Point cloud feature representation.

Representation for fea-

tures used in point cloud detection includes 2D-view images,

voxels, and raw points. By projecting point clouds into a 2D

bird’s eye view or front view, 2D-view-based 3D detectors can

intuitively ﬁt into a 2D image detection pipeline [

], [

However, 2D-view images could lose depth information [

where the localization accuracy of the detector is affected.

To efﬁciently acquire 3D spatial knowledge in large-scale

point clouds, the “voxelization” operation is leveraged to

partition unordered points into spatially and evenly distributed

voxels [

], [

]. After pooling interior features, those voxels

are fed into a sparse 3D convolution backbone [

] for

feature abstraction. Given an appropriate voxelization scale,

voxel-based representation is computationally efﬁcient, but the

quantization loss by voxelization is also inevitable [

]. Different

from the above methods, PointNet [

] and PointNet++ [

]

directly extract abstract features from raw points, which keeps

the integrity of spatial context in point clouds. However, the

point-based detectors are not cost-efﬁcient for large-scale data

[

]. As a trade-off between the voxel-based and point-based

methods, Point-voxel-based representations [

], [

] possess

the potential of fusing the high-efﬁcient voxels and accurate-

abstract points in feature abstraction.

Proposal architecture.

One-stage detectors [

], [

] directly

generate candidate BBoxes from the abstracted features. To

improve candidate BBoxes’ precision, two-stage detectors [

[

] reﬁne those BBoxes by region proposal network (RPN) and

tailor them into uniﬁed size by region of interest (RoI) pooling

before predicting output BBoxes. Compared with one-stage

detectors, two-stage ones [

] usually present more accurate

localization but intuitively, are more computationally time-

consuming.

B. Robustness Enhancement Solutions

Several attempts have been made to enhance the robustness of

point cloud detectors. In this paper, we select data augmentation

and denoising methods to study their effects on improving point

cloud detectors’ robustness against common corruptions. Data

augmentation [

] is an effective way of increasing the amount

of relevant data by slightly modifying existing data or newly

creating synthetic data from existing data. Data augmentation

on the point cloud [

], [

] provides detectors with a way to

be trained with a larger dataset and thus potentially obtain more

robust detectors. Different from data augmentation, denoising

[

], [

] serves as a pre-process to detect and remove spatial

outliers in point clouds, which can reduce the effects of noisy

point cloud data.

IV. PHYSICAL-AWARE ROBUSTNESS BENCHMARK FOR

POINT CLOUD DETECTION

We propose the ﬁrst robustness benchmark of point cloud de-

tectors against common corruption patterns. We ﬁrst introduce

different corruption patterns collected for this benchmark and

dataset in Section

IV-A

. Then we propose the evaluation metrics

used in our benchmark in Section

IV-B

. Finally, we introduce

the subject-object detection methods and robustness enhance-

ment methods selected for this benchmark in Section IV-C.

A. Physical-aware Corrupted Dataset Construction

After the literature investigation in Section

II-A

, we summa-

rize 25 corruption patterns in Table II and categorize them into

4 categories based on presentations of common corruptions:

weather, noise, density, and transformation. On the other hand,

we also divide common corruption patterns into the scene-

level and the object-level. As an initial effort, the dataset

covers representative but not all corruptions, and we encourage

continuous work with more diverse corruptions considered in

the future.

The simulation of corruptions implemented in the paper

mainly operates on the spatial locations and the reﬂection

intensity of points in the point cloud. Those point-targeting

operations are equivalent to the perturbations of the real-world

corruptions on the LiDAR point cloud and have been widely

utilized in the simulation-related studies, as in noise-related

[

], [

], density-related [

], [

[

], [

], and transformation-related [

], [

[

]. Next, We brieﬂy introduce each corruption pattern in the

following (refer to Appendix C for detailed implementations

and visualizations).

Weather corruptions:

LiDAR is sensitive to adversarial

weather conditions, such as rainy, snowy, and foggy [

]. Dense

droplets of liquid or solid water dim the reﬂection intensity

and reduce the signal-to-noise ratio (SNR) of received lights.

Floating droplets can also reﬂect and fool sensors with false

alarms. Both effects, in some cases, can signiﬁcantly affect

the detectors. To simulate three weather corruptions: {rain,

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1CommonCorruptionRobustnessofPointCloudDetectors:BenchmarkandEnhancementShuangzhiLi,ZhijieWang,FelixJuefei-Xu,QingGuo*,XingyuLi,andLeiMaAbstractObjectdetectionthroughLiDAR-basedpointcloudhasrecentlybeenimportantinautonomousdriving.Althoughachievinghighaccuracyonpublicbenchmarks,thestate-of-the-artd...

展开>> 收起<<

1 Common Corruption Robustness of Point Cloud Detectors Benchmark and Enhancement.pdf

共16页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 Common Corruption Robustness of Point Cloud Detectors Benchmark and Enhancement

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: