Fairness in Generative Modeling do it Unsupervised

2025-04-22 0 0 652.35KB 10 页 10玖币

侵权投诉

Fairness in Generative Modeling: do it Unsupervised!

M. Zameshina1,2, O. Teytaud2

1. LIGM, Univ Gustave Eiel, CNRS,

ESIEE, Paris

2. Facebook AI Research

France

{mzameshina,oteytaud}@fb.com

Fabien Teytaud

Univ. du Littoral Cote d’Opale

France

teytaud@univ-littoral.fr

Vlad Hosu

Univ. of Konstanz

Germany

vlad.hosu@uni-konstanz.de

Nathanael Carraz

Univ. d’Antananarivo

Madagascar

carraznathanael@live.fr

Laurent Najman

LIGM, Univ Gustave Eiel, CNRS,

ESIEE Paris

France

laurent.najman@esiee.fr

Markus Wagner

The University of Adelaide

Australia

markus.wagner@adelaide.edu.au

ABSTRACT

We design general-purpose algorithms for addressing fairness issues

and mode collapse in generative modeling. More precisely, to design

fair algorithms for as many sensitive variables as possible, including

variables we might not be aware of, we assume no prior knowledge

of sensitive variables: our algorithms use unsupervised fairness

only, meaning no information related to the sensitive variables is

used for our fairness-improving methods. All images of faces (even

generated ones) have been removed to mitigate legal risks.

KEYWORDS

Generative modeling, neural networks, fairness

ACM Reference Format:

M. Zameshina

1,2

, O. Teytaud

, Fabien Teytaud, Vlad Hosu, Nathanael Car-

raz, Laurent Najman, and Markus Wagner. 2022. Fairness in Generative

Modeling: do it Unsupervised!. In Proceedings of The Genetic and Evolution-

ary Computation Conference 2022 (GECCO ’22). ACM, New York, NY, USA,

10 pages. https://doi.org/10.1145/3520304.3528992

1 INTRODUCTION

Fairness has become prevalent at the intersection of ethics and

articial intelligence. Various forms of fairness are critical in online

media [

]. In the present paper, we consider fairness in the context

of generative modeling. More precisely, when modeling the proba-

bility distribution of faces, we typically observe that classes already

rare in the dataset become even rarer in the model. This phenome-

non is called Mode Collapse (MC) [

], and for sensitive variables,

it is one of the fairness issues. We propose tools based on statistical

reweighting (Sections 3.1 and 3.2) or on user feedback (Section 3.3)

for mitigating fairness issues (such as MC) in generative modeling.

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specic permission and/or a

fee. Request permissions from permissions@acm.org.

GECCO ’22, July 9–13, 2022, Boston, USA

ACM ISBN 978-1-4503-392-686. . . $15.00

https://doi.org/10.1145/3520304.3528992

1.1 Fairness

There are many facets to fairness. An algorithm may be considered

to be fair if its results are independent of some variables, particularly

for sensitive variables. Fairness [

] can be measured in terms of

separation, i.e., whether the probability of a given prediction, given

the actual value, is the same for all values of a sensitive variable.

The measurement can also be rephrased in terms of equivalent

false negative and true negative rates for all classes. A distinct

point of view is suciency: suciency holds if the probability of

actually belonging to a given group is the same for individuals from

that group and with dierent sensitive variables. Another point

of view is independence, i.e., when the prediction is statistically

independent of sensitive variables. Because it is known that the

many criteria for fairness are contradictory, it is necessary to design

criteria depending on the application. In the present paper, we

consider the case in which the goal is to preserve some frequencies.

Here, we consider the context of generative modeling. There is

a model trained on data, and we want this model to satisfy some

requirements on frequencies: for every class, we would like the

frequency to match some target frequency. Typically, for simplicity

in the present paper, the target frequency is the frequency in the

original dataset: however, the methods that we propose can be

adapted to other settings.

1.2 Generative modeling: fairness and mode

collapse

There are many measures of fairness, even in the specic case of

generative modeling [

]. The main criterion is whether all classes

are correctly represented. It is known that modeling frequently de-

creases the frequency of rare classes (i.e., mode collapse). In addition,

improving the image quality (for each image independently) aggra-

vates the diversity loss [

]. For a conditional generative model,

there is sometimes a ground truth. For example, in super-resolution,

we want the reconstructed image to match the sensitive variables

of the ground truth as closely as possible. This case became partic-

ularly critical since, e.g., [

]: a pixelized version of Barak Obama

can be “depixelized” to be that of a white man. [

] points out

the importance of fairness in the design of Generative Adversarial

Networks (GANs) before applying them, for example as an early

stage before supervised training. For addressing fairness issues, a

arXiv:2210.03517v1 [cs.NE] 6 Oct 2022

GECCO ’22, July 9–13, 2022, Boston, USA M. Zameshina1,2, O. Teytaud2, Fabien Teytaud, Vlad Hosu, Nathanael Carraz, Laurent Najman, and Markus Wagner

possibility is to increase editability: [

] disentangles latent variables

for separating editable and sensitive parts. Some works focus on

measuring fairness, for example, [

] uses causal methodologies

for measuring fairness in a counterfactual manner. Fairness can

be integrated directly into the training: [

] focuses on training a

GAN while protecting some variables.

1.3 Related work

[

] increases fairness in GANs in a supervised manner, i.e., given

the sensitive attributes. [

] targets and improves the fairness of

generated datasets. More similar to our work, [

] focuses on uncer-

tain sensitive variables, and [

] adds a bias in a GAN for mitigating

fairness issues. In the same fashion as the present work, [

] consid-

ers biasing a GAN without any retraining. We focus on generically

(i.e., independently of the application, data, and model) correcting

for potential bias present in a generative model, without knowing

the sensitive variables. The critical point is that sensitive variables

seem to often come up as a surprise: typically, people do not decide

to create an unfair algorithm actively. For example, in [

], the

designers of the faulty soap dispenser had just not imagined that

it might fail on black skins. Also, there may be relevant sensitive

variables that have not been initially considered: ethnicity or gen-

der are obvious sensitive variables, but aesthetics, body mass index,

social origin, or even the quality of the camera, geographical origin,

also matter.

Our goal is to have a generic correction independent of the

sensitive variables. The rst proposed method (Sections 3.1 and 3.2):

•

is not only for the fairness issues regarding sensitive vari-

ables: we also preserve diversity for more classical diversity

issues such as MC.

•does not need any retraining.

•

is more or less eective depending on cases but is designed

for (almost) never being detrimental (Section 4.2).

The second proposed method, which can be combined with the

previous one, proposes several generations and then lets the user

choose. Therefore, the user experience is modied: we expect the

user to assist the method by actively selecting relevant outputs.

Contrary to the generic method proposed above, which we will

implement thanks to reweighting, the new approach is not a drop-in

replacement. Moreover, this also does not need retraining.

1.4 Outline

Section 2 presents tools useful for the present work:

•

Use of Image Quality Assessment (IQA) to improve image

generation (Section 2.1): we connect this method to our re-

search by investigating how much this quality improvement

degrades fairness and how our proposed methods can miti-

gate such issues.

•

Reweighting via simple rejection sampling to improve fair-

ness and reduce MC when the variables used for computing

the reweighting values are correlated to the target sensitive

variables (Section 3.1).

Section 3 presents our proposed algorithms:

•

Reweighting as above, but with reweighed variables unre-

lated to target classes (Section 3.2). This second context is

Class A B C D

Frequency 17.8% 52.2% 17.5% 12.4%

Rank-correlation AvA -0.07 0.22 -0.11 0.06

Rank-correlation K512 -0.02 0.16 -0.08 0.02

Table 1: For four distinct classes of individuals A, B, C and

D (obtained using R), we present the rank-correlation of

the frequency of that class with AvA and K512 scores re-

spectively. AvA and K512 are visual quality estimators, deal-

ing with aesthetics and technical quality respectively. Vi-

sual quality assessment is a task fairly independent of se-

mantics and therefore should exhibit little if any ethnicity-

related biases. Dataset: faces generated by StyleGan2 (see

thispersondoesnotexist.com). Classes: ethnicity evaluated

by R (see R in Table 2). Observation: the biggest class has

the strongest, positive correlation.

therefore applicable when we do not know the target classes.

We propose a method which is a drop-in improvement of an

arbitrary generative model: as soon as we have features and

a generative model, we can apply Alg. 1.

•

Multi-objective optimization, through computation of sev-

eral solutions (typically Pareto fronts), to mitigate diversity

loss by providing more frequently at least one output of the

category desired/expected by the user.

Section 4 is a mathematical analysis. Section 5 presents experimen-

tal results.

2 PRELIMINARIES

2.1 Correlations image quality / sensitive

variables

We investigate the known correlation between the estimated quality

of an image and its membership to a frequent class [15, 24].

In order to demonstrate that this is easily observable, Table 1

presents the rank correlation between the aesthetic quality of an

image and the logit of that image for each of four classes of individ-

uals. We note that the most positively correlated class is the most

frequent. Our interpretation is that the technical quality of gener-

ated images is higher for the most frequent classes, inuencing the

aesthetics score.

2.2 Image generation: GAN, PGAN, and

EvolGan

Our work specializes in image generation, and in particular on faces.

We use the following image generation tools. Our baseline GAN is

Pytorch GAN Zoo ([

], based on progressive GANs (PGANs) [

]).

We also use EvolGan [

], which improves Pytorch GAN Zoo by

biasing the random choice of latent variables

𝑧

using K512 [

]. We

use three congurations of EvolGan, as it uses as a budget the

number of calls to the original GAN; the three congurations then

correspond to budgets 10, 20, and 40 (named

𝐸𝐺

10,

𝐸𝐺

20, and

𝐸𝐺

respectively). Besides the one based on a random search, EvolGan

has an option for CMA search [

] and PortfolioDiscrete-

(

)

(i.e.

the variant of the Discrete

(

)

-ES as in [

]): we also employ

these variants, with notation respectively EG-CMA-10 and EG-

(

)

-10 for budget 10, and similar variants for budget 20 and

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

FairnessinGenerativeModeling:doitUnsupervised!M.Zameshina1,2,O.Teytaud21.LIGM,UnivGustaveEiffel,CNRS,ESIEE,Paris2.FacebookAIResearchFrance{mzameshina,oteytaud}@fb.comFabienTeytaudUniv.duLittoralCoted’OpaleFranceteytaud@univ-littoral.frVladHosuUniv.ofKonstanzGermanyvlad.hosu@uni-konstanz.deNathanaelC...

展开>> 收起<<

Fairness in Generative Modeling do it Unsupervised.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Fairness in Generative Modeling do it Unsupervised

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: