Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval Zhaopeng Dou1 Zhongdao Wang1 Weihua Chen2

2025-04-29 0 0 1.4MB 18 页 10玖币

侵权投诉

Reliability-Aware Prediction via Uncertainty

Learning for Person Image Retrieval

Zhaopeng Dou1, Zhongdao Wang1, Weihua Chen2,

Yali Li1, and Shengjin Wang1B

1Department of Electronic Engineering and BNRist, Tsinghua University, China

2Machine Intelligence Technonlogy Lab, Alibaba Group

dcp19@mails.tsinghua.edu.cn,wgsgj@tsinghua.edu.cn

Abstract. Current person image retrieval methods have achieved great

improvements in accuracy metrics. However, they rarely describe the reli-

ability of the prediction. In this paper, we propose an Uncertainty-Aware

Learning (UAL) method to remedy this issue. UAL aims at providing

reliability-aware predictions by considering data uncertainty and model

uncertainty simultaneously. Data uncertainty captures the “noise” in-

herent in the sample, while model uncertainty depicts the model’s con-

ﬁdence in the sample’s prediction. Speciﬁcally, in UAL, (1) we propose

a sampling-free data uncertainty learning method to adaptively assign

weights to diﬀerent samples during training, down-weighting the low-

quality ambiguous samples. (2) we leverage the Bayesian framework to

model the model uncertainty by assuming the parameters of the network

follow a Bernoulli distribution. (3) the data uncertainty and the model

uncertainty are jointly learned in a uniﬁed network, and they serve as

two fundamental criteria for the reliability assessment: if a probe is high-

quality (low data uncertainty) and the model is conﬁdent in the pre-

diction of the probe (low model uncertainty), the ﬁnal ranking will be

assessed as reliable. Experiments under the risk-controlled settings and

the multi-query settings show the proposed reliability assessment is eﬀec-

tive. Our method also shows superior performance on three challenging

benchmarks under the vanilla single query settings. The code is available

at: https://github.com/dcp15/UAL

Keywords: Person Image Retrieval, Uncertainty, Reliability Assessment.

1 Introduction

Person image retrieval, also known as person re-identiﬁcation (ReID), aims at

associating a target person across non-overlapping camera views [45, 48, 62]. Al-

though current methods [21, 17, 32, 29, 44, 15, 4, 20] have achieved promising per-

formance on public benchmarks, they are reliability-agnostic, i.e., the prediction

3The online version contains supplementary material available at

https://doi.org/10.1007/978-3-031-19781-9 34.

arXiv:2210.13440v1 [cs.CV] 24 Oct 2022

2 Z. Dou et al.

with low reliability scores

with high reliability scores

Same ID

quality

confidence

quality

confidence

0.21

0.14

0.93

0.89 





reliability score

low

high

(a) (b)

Fig. 1. Observation and Motivation. (a) In the multi-query setting, low-quality query

images contain more ambiguous information. Reliability scores are required to down-

weight the weights of these queries. (b) The reliability score is related to two factors:

the quality of the sample and the model’s conﬁdence in the prediction of the sample.

of a probe can be generated anyway, but they rarely describe whether the pre-

diction is reliable. However, when people are identifying pedestrians, they not

only give the judgment result but also the reliability associated with it. Such a

reliability assessment mechanism is important in human decision-making [9] and

also essential in the ReID task. For example, in real scenarios, we often face the

problem of searching for a person through his/her multiple images (i.e., multi-

query settings), as a pedestrian is usually captured by several cameras and one

camera may capture a series of observations of the person. More generally, we

can add the retrieved positive ones into the query set for further comprehensive

retrieval. The quality of these query images varies, especially in complex scenes.

As shown in Fig. 1(a), low-quality query images contain more ambiguous infor-

mation. If we treat these query images equally, performance will degrade. At this

time, reliability scores are required to down-weight the low-quality query images.

However, current methods rarely consider the reliability assessment problem.

To remedy this issue, we propose a novel Uncertainty-Aware Learning (UAL)

method for the ReID task. UAL aims at not only giving an accurate prediction for

a sample but also providing a reliability score associated with it. The reliability

score is related to two factors, i.e., the quality of the sample and the conﬁdence

of the model in the prediction of the sample. These two factors are measured

by considering two types of uncertainty, i.e., data uncertainty and model uncer-

tainty. Data uncertainty captures the “noise” inherent in the observation and it

can describe the quality of the sample. Model uncertainty represents the model’s

“ignorance” and it can reﬂect the model’s conﬁdence in its prediction [24, 7].

In this paper, we propose a uniﬁed network to learn the data uncertainty

and the model uncertainty simultaneously. Speciﬁcally, ﬁrst, we project a sam-

ple into a Gaussian distribution in the latent space, the mean of the distribution

represents the feature, and the variance represents the data uncertainty. Dif-

ferent from [1, 52, 41] sampling feature vector from the Gaussian distribution,

we propose a sampling-free method to learn the data uncertainty and adaptively

down-weight low-quality ambiguous samples during training. Second, we leverage

the Bayesian framework to learn the model uncertainty, in which the parameters

of the network are assumed to follow the Bernoulli distribution. The model un-

Reliability-Aware Prediction for Person Image Retrieval 3

certainty is deﬁned as the dispersion degree of the feature vectors caused by the

distribution of the network parameter. Third, the data uncertainty and model

uncertainty are jointly learned in a uniﬁed network, and they serve as two cri-

teria to assess whether the result is reliable: as shown in Fig. 1(b), if a query

image is high-quality (low data uncertainty) and the model is conﬁdent in its

prediction of the query image (low model uncertainty), the ﬁnal result will be

assessed as reliable. Experiments under risk-controlled settings and multi-query

settings show the proposed reliability assessment is eﬀective.

The major contributions are summarized as: (1) We propose an uncertainty-

aware learning (UAL) method that can provide reliability-aware predictions

for the ReID task. (2) We introduce a sampling-free data uncertainty learning

method, which can improve the representation by explicitly inhibiting the neg-

ative impact of low-quality samples during training without any external clues.

(3) We propose a uniﬁed network to jointly learn data uncertainty and model

uncertainty. As far as we know, this is the ﬁrst work to apply data uncertainty

and model uncertainty to the ReID task simultaneously. (4) Experiments under

risk-controlled settings and multi-query settings show the reliability assessment

is eﬀective. Our method also shows superior performance in single query settings.

2 Related Work

Person ReID. Person ReID aims to associate a target person across diﬀerent

camera views. Existing methods can be broadly divided into two categories:

hand-craft methods [33, 51] and deep learning methods [30, 18, 20, 4, 15, 52]. The

key challenge is the large appearance variation caused by imperfect detection,

diﬀerent camera views, poses, and occlusions. To remedy these issues, several

works [12, 13, 35, 47, 8, 62, 14, 32, 21] are proposed to learn local features to cope

with the appearance variation. Although these methods have played a certain

role, they are reliability-agnostic. That is, the model can output a prediction for

a probe anyway, but it does not describe the reliability of the prediction.

Uncertainty in person ReID. There are mainly two types of uncertainty:

data uncertainty and model uncertainty [7, 23, 40, 24]. Many tasks have consid-

ered the uncertainty to improve the robustness and interpretability of models,

such as face recognition [41, 25, 1], semantic segmentation [19, 24] and Multi-view

learning [10]. In the ReID task, prior arts [52, 55, 43, 22] consider data uncertainty

to alleviate the problem of label noise or data outliers. D-Net [52] maps each

person image as a Gaussian distribution in the latent space with the variance

indicating the data uncertainty. PUCNN [43] extends the data uncertainty in

D-Net into the part-level feature. UNRN [55] incorporates the uncertainty into

a teacher-student framework to evaluate the reliability of the predicted pseudo

labels for unsupervised domain adaptive (UDA) person ReID. The uncertainty is

estimated as the inconsistency of these two models in terms of their predicted soft

multi-labels. UMTS [22] designs an uncertainty-aware knowledge distillation loss

to transfer the knowledge of the multi-shots model into the single-shot model.

Among these methods, the most relevant method to ours is D-Net [52]. Com-

4 Z. Dou et al.

pared to D-Net, our data uncertainty learning method is sampling-free, which

can explicitly suppress the ambiguous information contained in low-quality sam-

ples. We jointly learns the data uncertainty and the model uncertainty, which

can utilize the complementary information provided by them during training.

3 Methodology

The reliability score is related to two factors: the quality of the sample and the

conﬁdence of the model in its prediction, which are measured by data uncertainty

(Sec. 3.1) and model uncertainty (Sec. 3.2), respectively. They are incorporated

into a uniﬁed network (Sec. 3.3) for joint learning. Two settings (risk-controlled

and multi-query settings) are proposed to verify the eﬀectiveness in Sec. 3.4.

3.1 Learning Data Uncertainty

Data uncertainty captures the “noise” inherent in the observation. It can reﬂect

the quality of the sample, which is an essential factor in the reliability assessment.

Prior method. Prior art D-Net [52] considers the data uncertainty by map-

ping a sample xas a Gaussian distribution in the latent space,

p(z|x) = N(z;µ,σ2I) (1)

where µand σ2are the mean and variance vectors. µis the feature vector and

σ2refers to the data uncertainty of x. Then, they sample features from p(z|x)

by re-parameterization trick [27]: z′=µ+ϵσ,ϵ∼ N (0,I). The sampled z′

are utilized for vanilla cross-entropy loss Lce. To prevent the trivial solution of

variance decreasing to zero, a regularization term Lfu is added to constrain the

entropy of N(µ,σ2I) to be larger than a constant. The ﬁnal loss function is,

L=Lce +λLfu (2)

where λis the hyper-parameter to balance Lce and Lf u. Although this method

can capture the data uncertainty, there are two limitations: (1) it is sampling-

based, i.e., the feature is sampled from the Gaussian distribution during training,

which makes the optimization more diﬃcult because each iteration optimizes

only one point in the distribution, rather than entire distribution. (2) the ob-

jective does not explicitly distinguish samples with diﬀerent data uncertainty. It

is unclear how data uncertainty aﬀects feature learning. To mitigate these two

issues, we propose a sampling-free method to learn the data uncertainty and

explicitly adjust the attention to the samples according to their quality.

Our sampling-free data uncertainty learning method. We project a

sample xinto a Gaussian distribution N(µ, σ2I) in the latent space. Then the

likelihood of xbelonging to class iis formulated by,

p(x|y=i)∝1

(2πσ2)d

exp (−∥µ−wi∥2

2σ2) (3)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Reliability-AwarePredictionviaUncertaintyLearningforPersonImageRetrievalZhaopengDou1,ZhongdaoWang1,WeihuaChen2,YaliLi1,andShengjinWang1B1DepartmentofElectronicEngineeringandBNRist,TsinghuaUniversity,China2MachineIntelligenceTechnonlogyLab,AlibabaGroupdcp19@mails.tsinghua.edu.cn,wgsgj@tsinghua.edu.cn...

展开>> 收起<<

Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval Zhaopeng Dou1 Zhongdao Wang1 Weihua Chen2.pdf

共18页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval Zhaopeng Dou1 Zhongdao Wang1 Weihua Chen2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: