IDPL Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels

2025-05-08 0 0 1.51MB 13 页 10玖币

侵权投诉

IDPL: Intra-subdomain adaptation adversarial

learning segmentation method based on

Dynamic Pseudo Labels

Xuewei Li1,2,3[0000−0002−5330−7298], Weilun Zhang4, Jie Gao1,2,3, Xuzhou

Fu1,2,3, and Jian Yu1,2,3( )

1College of Intelligence and Computing, Tianjin University, Tianjin, China.

2Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China.

3Tianjin Key Laboratory of Advanced Networking, Tianjin, China.

{lixuewei,gaojie,fuxuzhou,yujian}@tju.edu.cn

4Tianjin International Engineering Institute, Tianjin University, Tianjin, China.

zhangweilun@tju.edu.cn

Corresponding Author

Abstract. Unsupervised domain adaptation(UDA) has been applied to

image semantic segmentation to solve the problem of domain oﬀset. How-

ever, in some diﬃcult categories with poor recognition accuracy, the seg-

mentation eﬀects are still not ideal. To this end, in this paper, Intra-

subdomain adaptation adversarial learning segmentation method based

on Dynamic Pseudo Labels(IDPL) is proposed. The whole process con-

sists of 3 steps: Firstly, the instance-level pseudo label dynamic genera-

tion module is proposed, which fuses the class matching information in

global classes and local instances, thus adaptively generating the optimal

threshold for each class, obtaining high-quality pseudo labels. Secondly,

the subdomain classiﬁer module based on instance conﬁdence is con-

structed, which can dynamically divide the target domain into easy and

diﬃcult subdomains according to the relative proportion of easy and

diﬃcult instances. Finally, the subdomain adversarial learning module

based on self-attention is proposed. It uses multi-head self-attention to

confront the easy and diﬃcult subdomains at the class level with the help

of generated high-quality pseudo labels, so as to focus on mining the fea-

tures of diﬃcult categories in the high-entropy region of target domain

images, which promotes class-level conditional distribution alignment be-

tween the subdomains, improving the segmentation performance of diﬃ-

cult categories. For the diﬃcult categories, the experimental results show

that the performance of IDPL is signiﬁcantly improved compared with

other latest mainstream methods.

Keywords: unsupervised domain adaptation (UDA) ·semantic segmen-

tation ·diﬃcult category ·dynamic pseudo labels ·intra-subdomain ad-

versarial learning

arXiv:2210.03435v2 [cs.CV] 20 Oct 2022

2 X. Li et al.

1 Introduction

The domain shift problem in UDA manifests itself as the inter-domain discrep-

ancy problem between synthetic images and real images in semantic segmenta-

tion task, making diﬃcult to generalize the model trained with synthetic data

to real data. Previous UDA methods [1,2] have done a lot of works to reduce

the domain shift, but there are still some problems with existing methods: First

of all, most of the self-training methods based on pseudo labels select a ﬁxed

threshold to ﬁlter all pseudo labels with high conﬁdence [3,4]. The model pay

more attention to the conditional distribution alignment of the samples in easy

categories between the two domains, resulting in the samples in diﬃcult cate-

gories with high entropy may be discarded; Moreover, most methods adopt the

strategy of global threshold [5,6], ignoring the separate consideration of diﬀer-

ent classes, which increases the risk of class imbalance in the generated pseudo

labels.These problems lead to low accuracy of generated pseudo labels.

In addition, to alleviate the distribution gap between data within the target

domain, the unsupervised intra-domain adaptation methods are proposed. For

example, UIDA [7] considers the global feature map of samples during adversarial

training between the easy and diﬃcult subdomains. However, not all spatial

regions maintain high entropy for diﬃcult samples. Using global features for

adversarial learning, the feature extraction of diﬃcult samples in low-entropy

regions may be aﬀected, causing negative transfer. Therefore it is not reliable to

use the global entropy of the predicted probability maps to divide easy/diﬃcult

subdomains. The results are shown in Fig. 1, for ”diﬃcult” categories (such as

”pole” in purple boxes, ”bike” in green boxes, and ”sidewalk” in brown boxes),

this method does not work well.

Inspired by UIDA [7], this paper proposes a more stable domain adap-

tation method to achieve intra-subdomain adversarial training, namely Intra-

subdomain adaptation adversarial learning method based on Dynamic Pseudo

Labels (IDPL). The method consists of 3 parts: Firstly, in order to improve

the pseudo labels quality of intra-domain adversarial learning, the instance-level

pseudo labels dynamic generation module is proposed. The threshold is dynami-

cally adjusted for diﬀerent semantic classes of each image, so that the model pays

more attention to the high-entropy regions in the image; Then, the subdomain

classiﬁer module based on instance conﬁdence is constructed to realize the hierar-

chical division from easy instances/diﬃcult instances to easy subdomain/diﬃcult

subdomain; Finally, on the basis of the divided two subdomains, the subdomain

adversarial learning module based on self-attention is constructed, and the mul-

tiple discriminator head structure is introduced to mine the class information

contained in the high-entropy or entropy ﬂuctuation regions, so as to more ac-

curately narrow the intra-domain diﬀerences. As shown in Fig. 1, the pseudo

labels of IDPL are signiﬁcantly better than the comparison method, especially

in these ”diﬃcult” categories.

The main contributions of this paper are summarized as follows:

(1)The instance-level pseudo labels dynamic generation module is proposed,

which aims to dynamically adjust the global threshold of each class according to

IDPL 3

Image ground truth UIDA ours

Fig. 1: Comparison of pseudo label results between IDPL and similar method.

the instance prediction of each class, fuse local and global information to guide

the model to generate high-quality pseudo labels.

(2)The intra-subdomain adversarial learning module based on instance con-

ﬁdence is proposed. The conﬁdence coeﬃcient is constructed according to the

relative proportion of easy and diﬃcult instances, thus the target domain can

be accurately divided into image-level easy and diﬃcult subdomains; Guide the

model to generate subdomain invariant features through intra-domain adversar-

ial training at the semantic class level, so that the model pays more attention

to the more diﬃcult categories, and the inﬂuence of concentrated high-entropy

regions on segmentation results can be alleviated more eﬀectively.

2 Method

The overall framework is shown in Fig. 2. The method can be divided into 3

steps. Firstly, momentum is introduced for each instance to gradually update

the threshold for each class, generating high-quality pseudo labels. Secondly, the

conﬁdence of each instance is calculated, and they are classiﬁed, then the target

domain is split according to the relative proportion of the two types of instances.

Thirdly, the self-attention heads are used to apply self-attention to each class

of the two subdomains separately, and the self-attention maps are used as the

weight of each class to conduct inter-class subdomain adversarial training.

2.1 Instance-level pseudo labels dynamic generation module(PLDG)

First, the pre-trained segmentation model G is used for preliminary prediction

of the target domain image, and the predicted probability maps MT=G(XT)

is obtained. Then, softmax is used to calculate the class probability prediction

of each pixel position (ht, wt) in the image XT, as shown in Eq. (1):

pxt

i,ht,wt=em(ht,wt,i)

j=1 em(ht,wt,j)

, mt∈Mt(1)

Where irepresents a speciﬁc semantic class, i∈ {0,1, . . . , C}; The number with

the largest value along the channel direction of a certain pixel position (ht, wt)

represents the class prediction probability of this pixel, and the corresponding

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

IDPL:Intra-subdomainadaptationadversariallearningsegmentationmethodbasedonDynamicPseudoLabelsXueweiLi1;2;3[0000000253307298],WeilunZhang4,JieGao1;2;3,XuzhouFu1;2;3,andJianYu1;2;3()1CollegeofIntelligenceandComputing,TianjinUniversity,Tianjin,China.2TianjinKeyLaboratoryofCognitiveComputingandApplicati...

展开>> 收起<<

IDPL Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

IDPL Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: