Cloud removal Using Atmosphere Model_2

2025-04-27 0 0 3.48MB 35 页 10玖币

侵权投诉

arXiv:2210.01981v1 [cs.CV] 5 Oct 2022

Cloud removal Using Atmosphere Model

Yi Guoa,∗, Feng Lib, Zhuo Wangb

aCentre for Research in Mathematics and Data Science, School of Computing,

Engineering and Mathematics, Western Sydney University, Parramatta, NSW 2150,

Australia

bQian Xuesen Laboratory of Space Technology, Beijing 100094, China

Abstract

Cloud removal is an essential task in remote sensing data analysis. As the im-

age sensors are distant from the earth ground, it is likely that part of the area

of interests is covered by cloud. Moreover, the atmosphere in between cre-

ates a constant haze layer upon the acquired images. To recover the ground

image, we propose to use scattering model for temporal sequence of images

of any scene in the framework of low rank and sparse models. We further de-

velop its variant, which is much faster and yet more accurate. To measure the

performance of diﬀerent methods objectively, we develop a semi-realistic sim-

ulation method to produce cloud cover so that various methods can be quan-

titatively analysed, which enables detailed study of many aspects of cloud

removal algorithms, including verifying the eﬀectiveness of proposed models

in comparison with the state-of-the-arts, including deep learning models, and

addressing the long standing problem of the determination of regularisation

parameters. The latter is companioned with theoretic analysis on the range

of the sparsity regularisation parameter and veriﬁed numerically.

Keywords:

Robust Principal Component Analysis, Sparse Models, Scattering Model,

Deep Learning

∗Corresponding author

Email address: y.guo@westernsydney.edu.au (Yi Guo)

Preprint submitted to Pattern Recognition October 6, 2022

1. Introduction

In this paper, we concern about the satellites imagery. As the imaging

sensors are deployed kilometres above the earth ground, clouds usually ap-

pear in the acquired images. The clouds are nuisance for data analysis tasks.

It is desirable to remove the cloud totally to recover clean ground scene,

which gives rise to cloud removal. Due to the versatility of remote sensing

imagery, cloud removal methods have to align to the characteristics of the

sensors, for example, multiple channels or single band. Meanwhile, the plat-

form is a decisive factor for the design of the algorithm, for example, the

computation limitation and power consumption restriction. Furthermore,

the analysis tasks after cloud removal has some inﬂuence as well. So one has

to consider all possible contributing factors in the modelling process.

Our data is single band satellites images of the same scene sampled from

diﬀerent time points which are subjected to light to moderate cloud covering

randomly at various regions. The aim is to recover images without cloud, i.e.

the clear images revealing the ground scene so that subsequent analysis can

be performed reliably, for example, object detection and tracking. Therefore

the ﬁdelity is the most important factor to be considered, in other words,

the recovered must be as close as possible to the truth, not just simply

“visually ﬁt” (look plausible from afar). Unfortunately, there is no objective

assessment except visual checking, and one of the goals of this paper is to ﬁll

this gap.

We focus on non-deep-learning based methods for cloud removal, although

latest deep learning methods were used as contenders in our empirical studies

subject to code availability, for example [1] and [2]. The reason for this is

that the ﬁdelity of the recovered images is a concern for deep learning based

methods. The workﬂow of these methods consists of two steps. The ﬁrst is

to identity cloud covered areas and remove them. The second is to apply

generative models to ﬁll the removed pixels. Generalised adversial networks

(GAN) based models are popular choice for image completion. However,

the working mechanism of GAN and its variants, heavily relies on the train-

ing data on which the distribution is modelled by transforming a speciﬁed

random distribution, e.g. uniform distribution or multivariate Gaussian dis-

tribution. Essentially, GAN is some sort of density estimator. Then the

question is, what if the scene that the satellite sampled never appears in the

training data? GAN will certainly generate something for the missing areas

but will not be able to stretch outside its modelled distribution even it is

conditioned on some posterior. Therefore we consider other alternatives, for

example, temporal mosaicing [3, 4]. Although enforcing spatial smoothness

is the most time consuming component, the ﬁdelity can be reassured that no

“alien pixels” will be inserted into the images like GAN based methods do.

Another possibility is matrix completion methods for missing pixel ﬁlling,

for example, [5] and its later development [6]. The main model behind these

methods is the low rank robust principal component analysis [7] coming from

a long development of robust PCA (RPCA) [8, 9] that is the eﬀorts to im-

prove the robustness of the linear PCA model by reducing the sensitivity to

outliers. The elegance of RPCA comparing to its peers is the simplicity in its

formation as well as its theoretical guarantee for the recovery of the low rank

signals and sparse noise. The application of RPCA implies that the observed

images are the summation of low rank ground images and sparse cloud cover

images (images with cloud only without background). It makes sense for such

arrangement assuming that the ground scene changes little after excluding

misalignment and geometric distortion, and clouds cover only small portion

of the scene. The low rank condition on ground component signals the way of

ﬁlling missing pixels and hence RPCA has better interpretability than GAN

methods.

It seems that the aforementioned two-step workﬂow should be able to be

consolidated to a single one using RPCA. Nonetheless this two-step strategy

was still adopted for no obvious reason, in which RPCA is only used for cloud

identiﬁcation and a low rank matrix completion follows after those cloud

aﬀected areas masked out. Two questions remains though. Firstly, where

is the atmosphere modelled in the image data?The atmosphere is reﬂected

as a thin haze layer in the acquired images which may not be negligible.

Secondly, is the simple additive model in RPCA really the right description

of the physics? Apparently not. The most realistic model so far is the so-

called atmosphere scattering model [10] for satellite images. Therefore one

should build atmospheric aﬀect into the model for cloud removal and ground

images recover.

2. Models considering atmosphere eﬀects

Before presenting proposed ones, we ﬁrst describe RPCA based methods

here in the setting of imagery applications. Let Ii∈Rd1×d2be the i-th

sampled image of size d1×d2and i= 1,...,n;D= [vec(I1), . . . , vec(In)]

where vec(X) is the vectorisation of matrix Xto be a column vector, and

hence D∈Rd×n(d=d1d2). The RPCA model shared in [5, 6] is the

following,

min

L,C kLk∗+λkCk1(1)

s.t. D=L+C

where kXk∗is the nuclear norm of X, i.e. the summation of all singular

values of X, which is the convex envelope for matrix rank, kXk1is the ℓ1

norm of X,Lis the initial recovered ground images, Cis the cloud cover

images, and both are the same size as D.λis the regularisation parameter

usually ﬁxed to be 1

√das recommended in [7]. By introducing group sparsity

(deﬁned by super-pixels) and alignment into (1), [6] claims slightly better

performance. After solving (1), both methods proceed to matrix completion

with the mask derived from Cas follows

min

B,S kLk∗+αkSΩk1+βkS¯

Ωk1(2)

s.t. D=B+S

where Ω is the mask matrix of size d×nwith 0’s for masked out elements

and 1’s for others, ¯

Ω is the negated version of Ω, i.e. ﬂipping 0’s and 1’s, and

SΩis the projection of Son Ω, i.e. masking out elements indicated by 0’s in

Ω. The ijth element in the mask matrix, [Ω]ij = 1 if [C]ij > γσ(vec(C)) and

[Ω]ij = 0 otherwise, where σ(v) is the standard deviation of vand γ∈[0,1] is

a pre-set ratio. Bis the ﬁnal recovered ground images, which are supposed

to be cloud free. Sis the noise. In implementation, γ= 0.8, α=0.1

√d

and β= 1. Both problems are convex with two blocks of variables. There

are many gradient projection based solvers/optimisers for them under the

ADMM framework [11]. They all work reasonably well for moderate size of

images, for example, d1=d2= 1024 and n= 7.

The critical step is in (1) where cloud cover Cis supposed to be separated.

Note that the decomposition of the observed data D=L+Creﬂects the basic

model assumption. As mentioned earlier, this departures from the reality by

ignoring atmosphere eﬀect. So instead of simple additive model we propose

to use atmosphere scattering [10], D=L◦(1 −C) + C, in the modelling,

and hence optimising the following

min

L,C kLk∗+λkCk1(3)

s.t. D=L◦(1 −C) + C

[L]ij ∈[0,1],[C]ij ∈[0,1]

where X◦Yis the element-wise product of matrix Xand Yof the same size.

In the above formulation, it is assumed that the pixels in observed images

are rescaled to [0,1], which is easily done by dividing the maximum digital

number of the sensor, but not the maximum of the observed values. Note

that (3) is no longer a convex problem as the equality condition is not aﬃne.

It is supposed to be much diﬃcult to solve on itself, let alone the boxed

conditions clamping the elements in both Land Cwithin [0,1]. Nonetheless,

there is still some strategies for the optimisation. Fore example, introducing

a dummy variable Xto untangle the interaction between Land C

min

L,C kLk∗+λkCk1(4)

s.t. D=X◦(1 −C) + C

L=X

[L]ij ∈[0,1],[C]ij ∈[0,1],[X]ij ∈[0,1]

and proceed with the normal ADMM. However, we observed that this does

not converge well enough to be practically useful. Instead, we employ lineari-

sation using primal accelerated proximal gradient method [12] for its ease in

handling entangled nuclear norm optimisation and stability. The Lagrange

of (3) with proximity is

L=kLk∗+λkCk1+hY, D −L◦(1 −C)−Ci(5)

+µ

2kD−L◦(1 −C)−Ck2

leading to

L=kLk∗+λkCk1+µ

2kD−L◦(1 −C)−C+Y

µk2

F(6)

by ignoring constants, where kXkFis the Frobenius norm of X,Y∈Rd×nis

Lagrangian parameters for the equality condition and µ≥0 is the proximity

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

arXiv:2210.01981v1[cs.CV]5Oct2022CloudremovalUsingAtmosphereModelYiGuoa,∗,FengLib,ZhuoWangbaCentreforResearchinMathematicsandDataScience,SchoolofComputing,EngineeringandMathematics,WesternSydneyUniversity,Parramatta,NSW2150,AustraliabQianXuesenLaboratoryofSpaceTechnology,Beijing100094,ChinaAbstractC...

展开>> 收起<<

Cloud removal Using Atmosphere Model_2.pdf

共35页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Cloud removal Using Atmosphere Model_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: