Generalised Likelihood Ratio Testing Adversaries through the Differential Privacy Lens Georgios Kaissis 1 2 4 Alexander Ziller1 4 Stefan Kolek Martinez de Azagra3 4 and

2025-05-06 0 0 677.19KB 15 页 10玖币

侵权投诉

Generalised Likelihood Ratio Testing Adversaries through

the Differential Privacy Lens

Georgios Kaissis *1, 2, 4, Alexander Ziller1, 4, Stefan Kolek Martinez de Azagra3, 4, and

Daniel Rueckert1, 2

1Artiﬁcial Intelligence in Medicine and Healthcare, Technical University of Munich

2Department of Computing, Imperial College London

3Mathematical Foundations of Artiﬁcial Intelligence, LMU Munich

4These authors contributed equally

Abstract

Differential Privacy (DP) provides tight upper

bounds on the capabilities of optimal adversaries,

but such adversaries are rarely encountered in prac-

tice. Under the hypothesis testing/membership in-

ference interpretation of DP, we examine the Gaus-

sian mechanism and relax the usual assumption

of a Neyman-Pearson-Optimal (NPO) adversary

to a Generalized Likelihood Test (GLRT) adver-

sary. This mild relaxation leads to improved pri-

vacy guarantees (see Figure 1 below), which we

express in the spirit of Gaussian DP and

(ε

δ)

-DP,

including composition and sub-sampling results.

We evaluate our results numerically and ﬁnd them

to match the theoretical upper bounds.

1 Introduction

Differential Privacy (DP) and its applications to ma-

chine learning (ML) have established themselves

as the tool of choice for statistical analyses on sensi-

tive data. They allow analysts working with such

data to obtain useful insights while offering objec-

tive guarantees of privacy to the individuals whose

data is contained within the dataset. DP guarantees

are typically realised through the addition of cali-

brated noise to statistical queries. The randomisa-

tion of queries however introduces an unavoidable

*Corresponding author e-mail: g.kaissis@tum.de

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

GLRT adversary

NPO adversary

Figure 1: Our results at a glance: A minimal re-

laxation of the threat model from an NPO adver-

sary (red curve) to a GLRT adversary leads to a

substantially more optimistic outlook on privacy

loss (blue curve) from

(ε

δ)=(

0.95, 10

−4)

-DP to

(

0.37, 10

−4)

-DP at

∆/σ=

0.5.

Pd/f

: Probability of

detection/false-positive.

“tug-of-war” between privacy and accuracy, the so-

called privacy-utility trade-off. This trade-off is unde-

sirable and may be among the principal deterrents

from the widespread willingness to commit to the

usage of DP in statistical analyses.

arXiv:2210.13028v1 [cs.CR] 24 Oct 2022

The main reason why DP is considered harmful

for utility is perhaps an incomplete understand-

ing of its very formulation: In its canonical deﬁni-

tion, DP is a worst-case guarantee against a very

powerful (i.e. optimal) adversary with access to

unbounded computational power and auxiliary in-

formation [17]. Erring on the side of security in

this way is prudent, as it means that DP bounds

always hold for weaker adversaries. However, the

privacy guarantee of an algorithm under realistic

conditions, where such adversaries may not ex-

ist, could be more optimistic than indicated. This

naturally leads to the question what the “actual”

privacy guarantees of algorithms are under relaxed

adversarial assumptions.

Works on empirical veriﬁcation of DP guarantees

[2, 6, 13] have recently led to two general ﬁndings:

The DP guarantee in the worst case is (almost)

tight, meaning that an improved analysis is

not able to offer stronger bounds on existing

algorithms under the same assumptions;

A relaxation of the threat model on the other

hand leads to dramatic improvements in the

empirical DP guarantees of the algorithm.

Motivated by these ﬁndings, we initiate an in-

vestigation into a minimal threat model relax-

ation which results in an “almost optimal” adver-

sary. Complementing the aforementioned empiri-

cal works, which instantiate adversaries who con-

duct membership inference tests, we assume a for-

mal viewpoint but retain the hypothesis testing

framework. Our contributions can be summarised

as follows:

•

We begin by introducing a mild formal relax-

ation of the usual DP assumption of a Neyman-

Pearson-Optimal (NPO) adversary to a Gener-

alised Likelihood Ratio Testing (GLRT) adver-

sary. We discuss the operational signiﬁcance

of this formal relaxation in Section 3;

•

In this setting, we provide tight privacy guar-

antees for the Gaussian mechanism in the

spirit of Gaussian DP (GDP) and

(ε

δ)

-DP,

which we show to be considerably stronger

than under the worst-case assumptions, espe-

cially in the high privacy regime.

•

We provide composition results and subsam-

pling guarantees for our bounds for use e.g. in

deep learning applications.

•

We ﬁnd that –contrary to the worst-case

setting– the performance of the adversary

in the GLRT relaxation is dependent on

the dimensionality of the query, with high-

dimensional queries having stronger privacy

guarantees. We link this phenomenon to the

asymptotic convergence of our bounds to an

ampliﬁed GDP guarantee.

•

Finally, we experimentally evaluate our

bounds, showing them to be tight against em-

pirical adversaries.

2 Prior Work

Empirical veriﬁcation of DP

: Several prior works

have investigated DP guarantees from an empirical

point-of-view. For instance, [6] utilised data poi-

soning attacks to verify the privacy guarantees of

DP-SGD, while [13] instantiate adversaries in a vari-

ety of settings and test their membership inference

capabilities. A similar work in this spirit is [5].

Formalisation of membership inference at-

tacks

: [16] is among the earliest works to formalise

the notion of a membership inference attack against

a machine learning model albeit in a black-box set-

ting, where the adversary only has access to pre-

dictions from a targeted machine learning model.

Follow-up works like [21, 2] have extended the

attack framework to a variety of settings. Recent

works by [15] or by [12] have also provided formal

bounds on membership inference success in a DP

setting.

Software tools and empirical mitigation strate-

gies

: Alongside the aforementioned works, a va-

riety of software tools has been proposed to audit

the privacy guarantees of ML systems, such as ML-

Doctor [11] or ML Privacy Meter [21]. Such tools

operate on the premises related to the aforemen-

tioned adversary instantiation.

Of note, DP is not the only technique to defend

against membership inference attacks (although

it is among the few formal ones). Works like [10,

19] have proposed so-called model adaptation strate-

gies, that is, methods which empirically harden the

model against attacks without necessarily offering

formal guarantees.

Gaussian DP, numerical composition and sub-

sampling ampliﬁcation

: Our analysis relies heav-

ily on the hypothesis testing interpretation of DP

and speciﬁcally Gaussian DP (GDP) [3], however

we present our privacy bounds in terms of the more

familiar Receiver-Operator-Characteristic (ROC)

curve similarly to [9]. We note that for the purposes

of the current work, the guarantees are identical.

Some of our guarantees have no tractable analytic

form, instead requiring numerical computations,

similar to [4, 22]. We make strong use of the duality

between GDP and privacy proﬁles for privacy ampli-

ﬁcation by subsampling, a technique described in

[1].

3 Background

3.1 The DP threat model

We begin by brieﬂy formulating the DP threat

model in terms of an abstract, non-cooperative

membership inference game. This will then allow us to

relax this threat model and thus present our main

results in a more comprehensible way. Throughout,

we assume two parties, a curator

and an adver-

sary

and will limit our purview to the Gaussian

mechanism of DP.

Deﬁnition 1

(DP membership inference game)

Un-

der the DP threat model, the game can be reduced to the

following speciﬁcations. We note that any added com-

plexity beyond the level described below can only serve to

make the game harder for Aand thus improve privacy.

The adversary

selects a function

f:X → Rn

where

is the space of datasets with (known)

global

-sensitivity

∆

and crafts two adjacent

datasets

and

such that

D:={A}

and

D0:={A

. Here,

are the data of two

individuals and

fully known

. We denote the

adjacency relationship by '.

The curator

secretly evaluates either

f(D)

f(D0)

and publishes the result

with Gaussian

noise of variance σ2Incalibrated to ∆.

The adversary

, using all available information,

determines whether

was used for comput-

ing y.

The game is considered won by the adversary if they

make a correct determination.

Under this threat model, the process of comput-

ing the result and releasing it with Gaussian noise is

the DP mechanism. Note that the aforementioned

problem can be reformulated as the problem of de-

tecting the presence of a single individual given the

output. This gives rise to the description typically

associated with DP guarantees: “DP guarantees

hold even if the adversary has access to the data of

all individuals except the one being tested”. The

reason for this is that, due to their knowledge of

the data and the function

can always “shift”

the problem so that (WLOG)

f(A) =

0, from which

it follows that

f(B) = ∆

(where the strict equality

is due to the presence of only two points in the

dataset and consistent with the DP guarantee).

More formally, the problem can thus be ex-

pressed as the following one-sided hypothesis test:

H0:y=Zvs. H1:y=∆+Z,Z∼ N(0, σ2)

(1)

and is equivalent to asking

to distinguish the

distributions

σ2)

and

N(∆

σ2)

based on a sin-

gle draw. The full knowledge of the two distribu-

tions’ parameters renders both hypotheses simple.

In other words,

is able to compute the following

log-likelihood ratio test statistic:

log Pr(y| N(∆,σ2))

Pr(y| N(0, σ2)) =1

2σ2|y|2−|y−∆|2,

(2)

which depends only on known quantities. We

call this type of adversary Neyman-Pearson-Optimal

(NPO) as they are able to detect the presence of the

individual in question with the best possible trade-

off between Type I and Type II errors, consistent

with the guarantee of the Neyman-Pearson lemma

[14]. As is evident from Equation

(2)

, the capabili-

ties of an NPO adversary are independent of query

dimensionality. Due to the isotropic properties of

the Gaussian mechanism, the ability to form the

full likelihood ratio allow

to “rotate the problem”

in a way that allows them to linearly classify the

output, which amounts to computing the multivari-

ate version of the

-test. We remark in passing that

this property forms the basis of linear discriminant

analysis, a classiﬁcation technique reliant upon the

aforementioned property. GDP utilises the worst-

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

GeneralisedLikelihoodRatioTestingAdversariesthroughtheDifferentialPrivacyLensGeorgiosKaissis*1,2,4,AlexanderZiller1,4,StefanKolekMartinezdeAzagra3,4,andDanielRueckert1,21ArticialIntelligenceinMedicineandHealthcare,TechnicalUniversityofMunich2DepartmentofComputing,ImperialCollegeLondon3MathematicalF...

展开>> 收起<<

Generalised Likelihood Ratio Testing Adversaries through the Differential Privacy Lens Georgios Kaissis 1 2 4 Alexander Ziller1 4 Stefan Kolek Martinez de Azagra3 4 and.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Generalised Likelihood Ratio Testing Adversaries through the Differential Privacy Lens Georgios Kaissis 1 2 4 Alexander Ziller1 4 Stefan Kolek Martinez de Azagra3 4 and

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: