COMPUTATIONAL PHOTOGRAPHY CS-413 - EPFL 1 Target Aware Poisson-Gaussian Noise Parameters Estimation from Noisy Images

2025-04-29 0 0 710.39KB 11 页 10玖币

侵权投诉

COMPUTATIONAL PHOTOGRAPHY, CS-413 - EPFL 1

Target Aware Poisson-Gaussian Noise Parameters

Estimation from Noisy Images

Étienne Objois Kaan Okumu¸s Nicolas Bähler

Supervisor: Majed El Helou, Ph.D.

Professor: Prof. Sabine Süsstrunk

Abstract—Digital sensors can lead to noisy results under many

circumstances. To be able to remove the undesired noise from

images, proper noise modeling and an accurate noise parameter

estimation is crucial. In this project, we use a Poisson-Gaussian

noise model for the raw-images captured by the sensor, as it ﬁts

the physical characteristics of the sensor closely. Moreover, we

limit ourselves to the case where observed (noisy), and ground-

truth (noise-free) image pairs are available. Using such pairs is

beneﬁcial for the noise estimation and is not widely studied in

literature. Based on this model, we derive the theoretical max-

imum likelihood solution, discuss its practical implementation

and optimization. Further, we propose two algorithms based on

variance and cumulant statistics. Finally, we compare the results

of our methods with two different approaches, a CNN we trained

ourselves, and another one taken from literature. The comparison

between all these methods shows that our algorithms outperform

the others in terms of MSE and have good additional properties.

Index Terms—digital imaging sensors, noise estimation, Poisson

noise, Gaussian noise, raw-data, ground-truth image, cumulant,

CNN, maximum-likelihood.

I. INTRODUCTION

Every image capturing system is inherently noisy. The

noise is inﬂuenced by different factors and different systems

have different noise characteristics. In our project, we pick a

model of noise having two components, one being a Poisson

distribution and the other one a Gaussian.

Roughly, capturing an image can be seen as the process of

counting the number of incident photons that hit a sensor pixel

during a given amount of time. More photons in a given inter-

val of time translates to more light and hence more intensity

for the pixel of the ﬁnal image. Hence, the Poisson distribution

is inherent to that discrete photon counting phenomenon. The

Poisson contribution in that context is commonly referred to

as photon shot noise. The second element of our noise model,

the Gaussian, is introduced by a collection of different error

factors like the quantum efﬁciency, the circuitry, unwanted

interactions between pixels, read out errors and many more.

Overall, all those error sources combined can be modeled with

a single Gaussian.

Our goal is to estimate the parameters of this noise model.

Knowing those values enables performing noise correction.

More precisely, from an observed noisy image, yreconstruct

the ground-truth image x. In our setting, we assume to have

access to both yand x. This assumption is reasonable in a

calibration setting where one can do long exposure times to

minimize the Poisson contribution and average over several

images to reduce the impact of the Gaussian noise part. Once

calibrated (i.e., having estimated the noise parameters) newly

captured images (without knowing the ground-truth) can be

corrected for the noise leading to better results.

Generally, there are two main approaches to noise esti-

mation today, either using statistical techniques and signal

processing or deep learning. The former involves more domain

speciﬁc knowledge.

The method presented by Foi et al. [1] follows the ideas

of the ﬁrst approach. Additionally, it uses a Poisson-Gaussian

noise model like we do but only uses observations of the noisy

signal, not the ground-truth x. Hence, the problem the authors

of [1] try to solve is inherently more difﬁcult than ours. In

our setting we have knowledge of both yand x, hence, this

advantage should enable us to achieve better performance.

On the other hand, deep learning is increasingly often

applied to all kinds of ﬁelds, noise estimation is no exception.

Speciﬁcally, Convolutional Neural Networks (CNN) that are

abundantly used in many image related tasks. Here, we are

not limited to using only ybut also xand maybe even |y−x|

In this project, we propose novel methods of noise esti-

mation while comparing their performance to different ap-

proaches. Further, we put those results into perspective by

providing the log-likelihood we derived for this problem.

In the case where both xand yare at hand, our ﬁndings

allow improving over the conventional methods.

For our method to work, we heavily rely on the knowl-

edge of the noise-free ground-truth image x. Further, we

are only working with grayscale images, but our methods

are extendable to multichannel images. Each channel’s noise

parameters might be different from each other, as each channel

is independent from any other. Additionally, we didn’t address

the issue of clipping, i.e., handling values that lay outside the

range of valid pixel intensities. For instance, intensity is given

by a value in [0., 1.]and any pixels’ intensity beyond this

interval should be clipped to it’s closest bound of the interval

in order for it a valid value. But clipping is introducing a

nonlinearity which makes all the derivations we make more

complex. For simplicity, we allowed values to exceed the range

and do not apply any clipping.

II. RELATED WORK

Denoising is one of the most fundamental tasks in image

restoration, with both theoretical impact and practical appli-

cations. Most classic denoisers, for instance PURE-LET [2],

arXiv:2210.12142v2 [eess.IV] 20 Dec 2022

COMPUTATIONAL PHOTOGRAPHY, CS-413 - EPFL 2

KSVD [3], WNNM [4], BM3D [5], and EPLL [6], require

knowledge of the noise level in the input test image. Deep

learning image denoisers that have shown improved empirical

performance [7], [8] also require knowledge of noise distri-

butions, if not at test time [9], then at least for training [10],

[11]. This is due to the degradation overﬁtting of deep neural

networks [12]. Noise modeling is thus important for denoisers

at test time, but also for acquisition system analysis and

dataset modeling for training these denoisers. Past research has

focused on modeling noise from noisy images without relying

on ground truth, i.e., noise-free, information [1]. Interesting

approaches, for example Sparse Modeling [13], Dictionary

Learning [14] or non-local image denoising methods like

SAFPI [15], have been developed to push overall denoising

performance. However, none of these methods allow easy use

of noise-free data when it is available. For Poisson-Gaussian

noise modeling, for example, both FMD [16] and W2S [17]

rely on a noise modeling method that does not consider ground

truth noise-free images [1]. Hence, our approach to model the

Poisson-Gaussian Image Noise (PoGaIN) distribution exploits

paired samples (noisy and noise-free images), which signiﬁ-

cantly improves the modeling accuracy. Our method is based

on the cumulant expansion, which is also used by other authors

to derive estimators for PoGaIN model parameters, but for

different input types, such as noisy image time series [18] or

single noisy images [19]. We lastly refer the reader to our

concise publication that sums up the essential elements of this

report [20].

III. THEORY

A. Poisson-Gaussian Modeling

The generic signal-dependent Poisson-Gaussian noise mod-

eling can be written as the following form :

y=1

aα+β, α ∼ P(ax), β ∼ N(0, b2)(1)

where xis the known ground-truth signal and yis the observed

signal. In our modeling, Poisson signal-dependent component

ηpand Gaussian signal-independent component ηgare deﬁned

as,

ηp=1

aα, ηg=β(2)

where these two components are assumed to be independent.

From the derivation given in Appendix A, the following

properties of the observed signal, y, can be found :

E[ηp] = x, V[ηp] = x

a(3)

Here, the fact that Poisson noise has signal-dependent charac-

teristics. On the other hand, the Gaussian noise has the con-

stant variance and mean, which makes it signal independent as

expected. Consequently, the following equation 4 is obtained.

E[y] = x, V[y] = x

a+b2(4)

Intuitively, it means that the average of the observed image

should be the ground-truth image, which justiﬁes the rea-

soning. From the variance equation, the fact that variance

is affected directly by aand bmakes it reasonable as they

represent the noises.

B. Raw-Data Modeling

Poisson-Gaussian model is properly matched with the nat-

ural characteristics of raw-data of digital imaging systems.

The Poisson noise models the signal-dependent part of errors,

which are caused by the discrete nature of the photon-counting

process. On the other hand, Gaussian noise models the signal

independent errors, such as electric and thermal noise.

The parameter of the Poisson noise, αis dependent on the

quantum efﬁciency of the sensor. The more the number of

photons to generate the electrons inside the sensor is, the less

the value of αis. According to experiments conducted by [21],

this Poisson noise effect can be the dominant contributor to

uncertainty in the raw data captured by high-performance

sensors. This justiﬁes the accuracy of the modeling for raw-

image of the sensors.

Analog gain which is the ampliﬁcation of the collected

charge in the digital imaging systems is another dependent

that affects both Poisson and Gaussian noise parameters. In

digital camera, it is controlled by ISO and/or exposure index

(EI) sensitivity settings. The larger the ISO number is, the

larger analog gain resulted in, which causes the ampliﬁcation

of the noises as well. This causes the decrease in SNR of

the captured raw-data, which means the noise increases. We

can conclude that both of the noise parameters can be highly

dependent on the analog gain.

In the case of the system with large photon counting

condition, Poisson noise can be approximated as Gaussian

noise as the following.

P(λ) = N(λ, λ)(5)

This approximation can be useful for deriving the solution

based only on mean and variance, which simpliﬁes the process

of proposing algorithms without using ground-truth image [1].

However, since we were also looking for the method that

uses the ground-truth image as input, this approximation is

not applied for the following sections. Another reason is that

this approximation results in the loss of information about the

statistics of the actual noise parameters. In other words, it

results in lossy projection of aand binto less dimensional

space, which is not desirable.

C. Maximum Likelihood Solution

When the noise modeling in equation 1 is applied to a

raw-data image, the following likelihood function of the pixel

intensity of an observed image can be achieved with the

derivation explained in Appendix A.

fy(yn|a, b, x) =

∞

k=0

(axn)k

k!b√2πexp −axn−(yn−k/a)2

2b2

(6)

where yis observed image, xis the ground-truth image and

nis the pixel index.

In order to propose a robust noise parameter estimation

algorithm, the optimality criterion is chosen to be the max-

imization of the likelihood function in 6 with respect to noise

parameters, aand b. The resulted solution of this optimality

COMPUTATIONAL PHOTOGRAPHY, CS-413 - EPFL 3

criterion is called as Maximum Likelihood solution. From the

derivation in A, the following solution is found.

ˆa, ˆ

b= arg max

a,b Y

∞

k=0

(axn)k

k!b√2πexp −axn−(yn−k/a)2

2b2

(7)

IV. IMPLEMENTED METHODS

A. Grid Search for ML Solution

Maximum Likelihood solution offers an accurate estimation

of the noise parameters in theory. However, for the practical

reasons, it’s hard to propose the algorithmic solution for the

maximization of the functional inside the ML solution. This

functional in equation 7 is analyzed and found to be non-

concave. Thus, gradient-based optimization algorithms cannot

be applied for this maximization problem. For the sake of

implementation of ML solution, the most naive method is

proposed to estimate the noise parameters aand b. This is also

possible, as we only have two parameters to be estimated.

As an implementation issue, exact calculation of likelihood

function is difﬁcult as it includes inﬁnity sum as seen in

equation 6. In order to approximately estimate it, a sufﬁciently

large value of kmax is chosen, and the following is applied:

fy(yn|a, b, x)≈

kmax

k=0

(axn)k

k!b√2πexp −axn−(yn−k/a)2

2b2

(8)

However, from the analysis of the non-concavity behavior

of the likelihood function, it’s found that it does not result in

sufﬁciently good results for the estimation of noise parameters.

In order to achieve the accurate results, very small step sizes

should be chosen. This makes the algorithm computationally

too expensive to be solved in practice, and it is justiﬁed by

the testings.

Therefore, in this project, we present and compare three

different methods from ML solution to estimate the parameters

of the POISSON-GAUSSIAN NOISE. The ﬁrst method is based

on the variance of the noisy image for each pixel value of

the real image. Then, we will present a method based on the

cumulant of the noisy image and the knowledge we have of

the real image. Finally, we implemented a basic convolution

neural network in order to compare our result.

B. VARIANCE

This method is based on the variance of the values of the

noisy image for a ﬁxed intensity of the real image. That is to

say, we take a pixel ifrom xof intensity xi, then for every

jsuch that xj=xi, we have yj∼P(axi)

a+N(0, b2). Thus,

if we denote Yi={yk:xk=xi}, we have V[Yi]≈xi

a+b2.

We can calculate this variance with each distinct value of xi.

In our case, images are saved in 8-bits, thus we only have

256 unique different values of xi. Moreover, as we know the

theoretical mean of Yiis xi, we can calculate the variance

using :

V[Yi] = 1

|Yi|X

yk∈Yi

(yk−xi)2(9)

Finally, to obtain the estimation of a, b is :

ˆa, ˆ

b= arg min

a,b X

(V[Yi]−xi

a−b2)2(10)

Note that in equation 10, the same point (xi,V[Yi]) is

present |Yi|times. This is because we found better result

using this bias. This method has multiple default, ﬁrst it is

not unbiased, then it works best on images with a small

amount of unique pixel intensities but an important difference

between the minimum and maximum intensity. Also, because

it is biased, this method can be tuned to be better (for instance,

the importance of each terms on the right side of equation 10

can be modiﬁed so that higher values of xihas a smaller

weight).

C. CUMULANT

This method uses the cumulant expansion of the noisy

image. In this section, instead of seeing xand yas images,

we see xand yas samples from a distribution where x∼ X

and y∼ Y such that :

P[x=xi] = |{k:xk=xi}|

where ncorrespond to the number of sample (i.e., the size of x

and y). Then we can deﬁne Yas the distribution of POISSON-

GAUSSIAN NOISE over the distribution X. Formally:

Y ∼ P(aX)

a+N(0, b2)(11)

We then use the equation 12 calculated in appendix B to get

the cumulant of Yas a system of two equations :

κ2=x

a+x2−x2+b2

κ3=x3−3x2x+ 2x3+ 3x2

a−3x2

a+x

(12)

where xkj= ( 1

nPixk

i)j. Equations 12 forms a system of two

equations with two variables : aand b, the parameters of the

noise. This method beneﬁts being unbiased for ﬁnding κ2,3,

some extra-calculation can be made so make ˆaand ˆ

bunbiased.

D. CNN

For the sake of comparison with our methods presented

above, we implement a convolutional regression network

trained to predict aand b. It uses fairly standard layers, but

isn’t inspired by any particular architecture. For optimization,

we used an Adam [22] optimizer and for the loss we picked

Mean Absolute Percentage Error, which is given by

100

n=1 

vpred,n −vreal,n

vreal,n (13)

where vreal,n are the predictions made by the model and

vreal,n the ground-truth values. This speciﬁc loss is nice

because it is normalized by the real value, hence errors for

big values are not over penalized.

The detailed architecture of the CNN can be found in table I.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

COMPUTATIONALPHOTOGRAPHY,CS-413-EPFL1TargetAwarePoisson-GaussianNoiseParametersEstimationfromNoisyImagesÉtienneObjoisKaanOkumu¸sNicolasBählerSupervisor:MajedElHelou,Ph.D.Professor:Prof.SabineSüsstrunkAbstractDigitalsensorscanleadtonoisyresultsundermanycircumstances.Tobeabletoremovetheundesirednoise...

展开>> 收起<<

COMPUTATIONAL PHOTOGRAPHY CS-413 - EPFL 1 Target Aware Poisson-Gaussian Noise Parameters Estimation from Noisy Images.pdf

共11页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

COMPUTATIONAL PHOTOGRAPHY CS-413 - EPFL 1 Target Aware Poisson-Gaussian Noise Parameters Estimation from Noisy Images

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: