ASURVEY OF IDENTIFICATION AND MITIGATION OF MACHINE LEARNING ALGORITHMIC BIASES IN IMAGE ANALYSIS Laurent Risser12 Agustin Picard23 Lucas Hervier4 Jean-Michel Loubes12

2025-05-06 0 0 1.21MB 19 页 10玖币

侵权投诉

ASURVEY OF IDENTIFICATION AND MITIGATION OF MACHINE

LEARNING ALGORITHMIC BIASES IN IMAGE ANALYSIS

Laurent Risser1,2, Agustin Picard2,3, Lucas Hervier4, Jean-Michel Loubes1,2

1Institut de Mathématiques de Toulouse (UMR 5219), CNRS, Université de Toulouse, F-31062 Toulouse, France

2Artiﬁcial and Natural Intelligence Toulouse Institute (ANITI), Toulouse, France

3Scalian, Labège, France

4Institut de Recherche Technologique (IRT) Saint Exupéry, Toulouse, France

ABSTRACT

The problem of algorithmic bias in machine learning has gained a lot of attention in recent years due

to its concrete and potentially hazardous implications in society. In much the same manner, biases can

also alter modern industrial and safety-critical applications where machine learning are based on high

dimensional inputs such as images. This issue has however been mostly left out of the spotlight in the

machine learning literature. Contrarily to societal applications where a set of proxy variables can be

provided by the common sense or by regulations to draw the attention on potential risks, industrial

and safety-critical applications are most of the times sailing blind. The variables related to undesired

biases can indeed be indirectly represented in the input data, or can be unknown, thus making them

harder to tackle. This raises serious and well-founded concerns towards the commercial deployment

of AI-based solutions, especially in a context where new regulations clearly address the issues opened

by undesired biases in AI. Consequently, we propose here to make an overview of recent advances

in this area, ﬁrstly by presenting how such biases can demonstrate themselves, then by exploring

different ways to bring them to light, and by probing different possibilities to mitigate them. We

ﬁnally present a practical remote sensing use-case of industrial Fairness.

Keywords Machine Learning, Trustworthy AI, Fairness, Computer Vision, Bias Detection, Bias Mitigation

1 Introduction

The ubiquity of Machine Learning (ML) models, and more speciﬁcally deep neural network (NN) models, in all sorts

of applications has become undeniable in recent years. From classifying images [

], detecting objects [

] and

performing semantic segmentation [

] to translating from one human language to another [

] and doing sentiment

analysis [

], the advances in different subﬁelds of ML can be attributed mostly to the explosion of computing power and

their ability to speed up the training process of artiﬁcial NNs. Most famously, AlexNet [

] allowed for an impressive

jump in performance in the challenging ILSVRC2012 image classiﬁcation dataset [

], also known as ImageNet,

permanently cementing deep convolutional NN (CNN) architectures in the ﬁeld of computer vision. Since then,

architectures have gotten more reﬁned [

], training procedures have gotten increasingly more complex [

], and

their performance and robustness have greatly improved as a consequence. Namely, the success of these deep CNN

models is related to their ability to treat high-dimensional and complex data such as images or natural language. The

impressive performance of NNs for machine learning tasks can be explained by the ability of their ﬂexible architecture

to capture meaningful information on various kinds of complex data and the fact that they are potentially composed of

millions of parameters.

However, this poses a major challenge: deciphering the reasoning behind the model’s predictions. For instance, typical

NN architectures for classiﬁcation or regression problems incrementally transform the representation of the input data

in the so-called

latent space

(or

feature space

) and then use this transformed representation to make their predictions,

as summarized in Fig. 1. Each step of this incremental data processing pipeline (or feature extraction chain) is carried

out by a so-called layer, which is mathematically a non-linear function (blue rectangle in Fig. 1). It is typically made

arXiv:2210.04491v1 [cs.LG] 10 Oct 2022

A survey of Identiﬁcation and mitigation of Machine Learning algorithmic biases in Image Analysis

Input Image

... ...

N.N. – part 1: Change

data representation

Latent

space N.N. – part 2 Prediction

Figure 1: General architecture of a neural network designed for classiﬁcation or regression tasks on images. It ﬁrst

non-linearly projects the input image information into a latent space, and then uses this transformed information for its

prediction.

of a linear transformation followed by a non-linear activation function [

], but more complex alternatives exist –

e.g. the residual block layers of ResNet models [

] or the self-attention layers [

] in transformer models. These ﬁrst

stages of the model (Fig. 1) often rely on the bottlenecking of the information that’s passing through it by sequentially

decreasing the size of the feature maps and applying non-linear transformations – e.g. the widely used ReLU activation

function [

]. To summarize, these ﬁrst stages project the input data into a latent space. Therefore, the neural network’s

data extraction pipeline is driven by the training data that were used to optimize its parameters. The second part of

the network (Fig. 1), which is standard for classiﬁers or regressors, is generally simpler to understand than the ﬁrst, as

it is often composed of matrix-vector products (often denoted as dense or fully-connected layers) followed by ReLU

activation functions. Consequently, it is mathematically equivalent to a piece-wise linear transformation [

]. More

importantly, these non-linear transformations depend on parameters that are optimized to make accurate predictions for

a particular task when training the NN.

Finally, it is worth emphasizing that the data transformation from the latent space to the NN’s output can be as complex

as in the ﬁrst part of the network Fig. 1 in models that are not designed for regression or classiﬁcation, such as e.g. the

unsupervised auto-encoder models [

] or U-Nets [

]. This makes their analysis and control even more complex than

in models following the general structure of Fig. 1.

The fact that neural networks are

black-box

models raises serious concerns for applications where algorithmic decisions

have life-changing consequences, for instance in societal applications or for high risk industrial systems. This issue has

motivated a substantial research effort over the last few years to investigate both explainability, and the creation and

propagation of bias in algorithmic decisions. An important part of this research effort has been made to explain the

predictions of black-box ML models [18, 19, 20, 21] or to detect out-of-distribution data [22, 23].

In this paper we will leverage the signiﬁcant work that has been made in the ﬁeld of

Fairness

, and study how it can

be extrapolated to industrial computer vision applications. Fairness in Machine Learning considers the relationships

between an algorithm and a certain input variable that should not play any role in the model’s decision from an ethical,

legal or technical point of view, but has a considerable inﬂuence on the system’s behavior nonetheless. This variable

is usually called the

sensitive variable

. Different deﬁnitions have been put in the statistical literature, each of them

considering speciﬁc dependencies between the sensitive variable and the decision algorithm. From a more practical

point of view, Fairness issues in Machine Learning manifest themselves in the shape of

undesired algorithmic biases

in the model’s predictions, such as according more bank mortgages to males than females for similar proﬁles or hiring

males rather than females for some speciﬁc job proﬁles, due to a majority presence of male individuals with the

corresponding proﬁle in the learning database. Hence, Fairness initially gained a lot of attention speciﬁcally in social

applications, with a large amount of articles speaking out about the different types of bias that ML algorithms amplify.

We refer for instance to the recent review papers of [24, 25] and references therein.

However, we want to emphasize that studies focusing on the presence of bias in more general industrial applications

based on complex data like images have mostly been left out of the spotlight. We intend to raise awareness about this

kind of problem for safety-critical and/or industrial applications, where trained models may be discriminating against

a certain group (or situation) in the form of a biased decision or diminished performance. We point out that a team

developing a NN-based application might simply be unaware of this behavior until the application is deployed. In

this case, speciﬁc groups of end-users may observe that it does not work as intended. A typical example of undesired

algorithmic bias in image analysis applications is the one that was made popular by the paper presenting the LIME

explainability technique [

]. Indeed, the authors trained a neural network to discriminate images representing wolves

A survey of Identiﬁcation and mitigation of Machine Learning algorithmic biases in Image Analysis

and huskies. Despite the NN’s reasonable accuracy, it was still basing itself off spurious correlations – i.e. the presence

or not of snow in the background – to decide whether the image contained a wolf. Another example that will be at

the heart of this paper is a blue veil effect in satellite images, which will be discussed in Section 5. When present,

these biases provide a shortcut for the models to achieve a higher accuracy score both in the training and test datasets,

although the logic behind the decision rules is generally false. This phenomenon is often modeled by the use of

confounding variables in statistics. Hence, they hinder these models’ performance when predicting a sample from

the discriminated group. This makes it completely clear that all harmful biases must be addressed in industrial and

safety-critical applications, as algorithmic biases might render the general performance guarantees useless in speciﬁc or

uncommon situations.

We make the following contribution in this survey:

• We summarize different types of bias and fairness deﬁnitions most commonly present for images.

•

We present a comprehensive review of methods to detect and mitigate biases with a particular focus on machine

learning algorithms devoted to images.

•

We identify open challenges and discussing future research direction around an industrial use case of image

analysis.

2 Fairness in Machine Learning

In this section, we will brieﬂy introduce the different deﬁnitions of Fairness that we will consider in this paper. In

particular, we will concentrate on statistical – or global – notions of Fairness that are the most popular among ML prac-

titioners. There exist other deﬁnitions based on causal mechanisms that provide a local measure of discrimination [

]

or [

] – and that play an important role in social applications, where discrimination can be assessed individually –, but

they are beyond the scope of this paper.

2.1 Deﬁnitions

Let

be the observed input images,

, the output variables to forecast and

, the sensitive variable that induces an

undesirable bias in the predictions (introduced in Section 1), which can be explicitly known or deduced from

(X, Y )

In a supervised framework, the prediction model

fθ

is optimized so that its parameters

minimize an empirical risk

R(Y, ˆ

, which measures the error of forecasting

, with

Y:= fθ(X)

. We will denote

L(Z)

the distribution of a

random variable Z.

An image is deﬁned as an application

X:K1×K27→ Rd

, where

and

are two compact sets representing the

pixel domain (

and

can also be considered for 3D or 3D+t images) and

is the number of image channels (e.g.

d= 3

for RGB images). We will consider 2D images with

d= 1

in the remainder of this section to keep the notations

simple. An image can thus be interpreted as an application mapping each of its coordinates

(i, j)

to a pixel intensity

value

X(i, j)

. Metadata, denoted here by

meta

and the sensitive variable

Bias can manifest itself in multiple ways depending on how the variable which causes the bias inﬂuences the different

distributions of the data and the algorithm.

Bias can originate from the mismatch between the different data distributions in the sense that small subgroups of

individuals have different distributions, i.e

L(Y, X|A)6=L(Y, X)

. This is the most common example that we can

encounter in image datasets. The ﬁrst consequence can be a

sampling bias

, and can discourage the model from learning

the particularities of the under-represented groups or classes. As a consequence, despite achieving a good average

accuracy on the test samples, the prediction algorithm may exhibit poor generalization properties when deployed on

real life applications with different subsets of distributions.

Another case emerges when external conditions that are not relevant for the experiment induce a difference in the

observed data’s labels in the sense that

L(Y|X, A)6=L(Y|X)

, therefore inadvertently encouraging models to learn

biased decisions (as in the Wolves versus Huskies example in [

]). This is the case when data is collected with labels

A survey of Identiﬁcation and mitigation of Machine Learning algorithmic biases in Image Analysis

inﬂuenced by a third unknown variable leading to

confounding bias

, or when the observation setting favors one class

over the other leading to

selection bias

. The sources of this bias may be related to observation tools, methods or

external factors as it will be pointed out later.

A third interesting case concerns the bias induced by the model itself, which is often referred to as

inductive bias

L(ˆ

Y|X, Y, A)6=L(ˆ

Y|X, Y )

. This opposes the world created by the algorithm –i.e. the distribution of the algorithm

outputs – to the original data. From a different point of view, bias can also arise when the different categories of the

algorithm outputs differ from the categories as originally labeled in the dataset – i.e.

L(Y|ˆ

Y , X, A)6=L(Y|ˆ

Y , X)

– a

condition that is often referred to as lack of sufﬁciency.

Finally, the two previous conditions can also be formulated by considering the distribution of the algorithm prediction

errors and their variability with respect to the sensitive variable:

L(`(Y, ˆ

Y)|X, A)6=L(`(Y, ˆ

Y)|X)

, where

Y×Y7→

`(ˆ

Y , Y )is the loss function measuring the error incurred by the algorithm by forecasting ˆ

Yin place of Y.

2.2 Potential causes of bias in Computer Vision

In practice, the above described situations may materialize through different causes in image datasets.

2.2.1 Improperly sampled training data

First, the bias may come from the data themselves, in the sense that the distribution of the training data is not the

ideal distribution that would reﬂect the desired behavior that we want to learn. Compared with tabular data, image

datasets can be difﬁcult to collect, store and manipulate due to their considerable size and the memory storage they

require. Hence, many of them have proven to lack diversity – e.g. because not all regions are studied (geographic

diversity), or not all sub-population samples are uniformly collected (gender or racial diversity). The growing use of

facial recognition algorithms in a wide range of areas affecting our society is currently debated. Indeed, they have

demonstrated to be a source of racial [

], or gender [

] discrimination. Besides, well-known datasets such as

CelebA [

], Open Images [

] or ImageNet [

] lack of diversity – as shown in [

] or [

] – resulting in imbalanced

samples. Thus, state-of-the-art algorithms are unable to yield uniform performance over all sub-populations. A similar

lack of diversity appear in the newly created Metaverse as pointed out in [

] creating racial bias. This encouraged

several researchers to design datasets that do not suffer from these drawbacks – i.e. preserving diversity – as illustrated

by the Pilot Parliament Benchmark (PPB) dataset [36] or in [37] or in Fairface dataset [38].

Combining diverse databases to get a sufﬁcient accuracy in all sub-populations is even more critical for high-stakes

systems, like those commonly used in Medicine. The fact that medical cohorts and longitudinal databases suffer from

biases has been long ago acknowledged in medical studies. The situation is even more complex in medical image

analysis for specialties such as radiology (National Lung Screening Trial, MIMIC-CXR-JPG [

], CheXpert [

]) or

dermatology (Melanoma detection for skin cancer, HAM10000 database [

]), where biased datasets are provided

for medical applications. Indeed, under-represented populations in some datasets lead to critical drop of accuracy, for

instance in skin cancer detection as in [42], [43] or for general research in medicine [44] and references therein.

The captioning of images is a relevant example where shortcoming of diversity hampers the quality of the algorithms’

predictions, and may result in biased forecasts as pointed out in [

] or in [

]. Therefore, it is of utmost importance to

include diversity (e.g geographic, social, ..) when building image datasets that will be used as reference benchmarks to

build and test the efﬁciency of computer vision algorithms.

2.2.2 Spurious correlations and external factors

The context in which the data is collected can also create spurious correlations between groups of images. Different

acquisition situations may provide different contextual information that can generate systematic artifacts in speciﬁc

kinds of images. For instance, confounding variables such as the snowy background in the Wolves versus Huskies

example of [

] (see Section 1) may add bias in algorithmic decisions. In this case, different objects in images may have

similar features due to the presence of a similar context, such as the color background, which can play an important

role in the classiﬁcation task due to spurious correlations. We refer to [

] for more references. This phenomenon is

also well known in biology where spectroscopy data are highly inﬂuenced by the ﬂuorescence methods as highlighted

in [

], which makes machine learning difﬁcult to use without correcting the bias. Different biases related to different

instruments of measures are also described for medical data in [49].

An external factor can also induce biases and shift the distributions. It is important to note that all images are acquired

using sensors and pre-processed afterwards, potentially introducing defects to the images. In addition, their storage

may require to compress the information they contain in many different ways. All this makes for a type of data with

a considerable variability depending on the quality of the sensors, pre-processing pipeline and compression method.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ASURVEYOFIDENTIFICATIONANDMITIGATIONOFMACHINELEARNINGALGORITHMICBIASESINIMAGEANALYSISLaurentRisser1;2,AgustinPicard2;3,LucasHervier4,Jean-MichelLoubes1;21InstitutdeMathématiquesdeToulouse(UMR5219),CNRS,UniversitédeToulouse,F-31062Toulouse,France2ArticialandNaturalIntelligenceToulouseInstitute(ANITI...

展开>> 收起<<

ASURVEY OF IDENTIFICATION AND MITIGATION OF MACHINE LEARNING ALGORITHMIC BIASES IN IMAGE ANALYSIS Laurent Risser12 Agustin Picard23 Lucas Hervier4 Jean-Michel Loubes12.pdf

共19页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

ASURVEY OF IDENTIFICATION AND MITIGATION OF MACHINE LEARNING ALGORITHMIC BIASES IN IMAGE ANALYSIS Laurent Risser12 Agustin Picard23 Lucas Hervier4 Jean-Michel Loubes12

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: