AltUB Alternating Training Method to Update Base Distribution of Normalizing Flow for Anomaly Detection Yeongmin Kim1 Huiwon Jang2 DongKeon Lee3 Ho-Jin Choi3

2025-04-30 0 0 689.37KB 9 页 10玖币

侵权投诉

AltUB: Alternating Training Method to Update Base Distribution of

Normalizing Flow for Anomaly Detection

Yeongmin Kim*1, Huiwon Jang*2, DongKeon Lee 3, Ho-Jin Choi 3

1School of Freshman, KAIST

2Department of Mathematical Science, KAIST

3School of Computing, KAIST

(cytotoxicity8 / huiwoen0516 / hagg30 / hojinc)@kaist.ac.kr

Abstract

Unsupervised anomaly detection is coming into the spotlight

these days in various practical domains due to the limited

amount of anomaly data. One of the major approaches for

it is a normalizing ﬂow which pursues the invertible transfor-

mation of a complex distribution as images into an easy dis-

tribution as N(0, I). In fact, algorithms based on normalizing

ﬂow like FastFlow and CFLOW-AD establish state-of-the-art

performance on unsupervised anomaly detection tasks. Nev-

ertheless, we investigate these algorithms convert normal im-

ages into not N(0, I)as their destination, but an arbitrary nor-

mal distribution. Moreover, their performances are often un-

stable, which is highly critical for unsupervised tasks because

data for validation are not provided. To break through these

observations, we propose a simple solution AltUB which in-

troduces alternating training to update the base distribution

of normalizing ﬂow for anomaly detection. AltUB effectively

improves the stability of performance of normalizing ﬂow.

Furthermore, our method achieves the new state-of-the-art

performance of the anomaly segmentation task on the MVTec

AD dataset with 98.8% AUROC.

1 Introduction

Automated detection of anomalies has become an important

area of computer vision in a variety of practical domains,

including industrial (Bergmann et al. 2019), medical (Zhou

et al. 2020) ﬁeld, and autonomous driving systems (Jung

et al. 2021). The essential goal of anomaly detection (AD)

is to classify whether an object is normal or abnormal, and

localize the regions of an anomaly if the object is classiﬁed

as an anomaly (Wu et al. 2021). However, it is challenging

to obtain a large number of abnormal images in real-world

problems.

An unsupervised anomaly detection task has been intro-

duced to address this imbalance in the dataset due to the lack

of defected samples. To do so, it aims at detecting defection

by using only the non-defected samples. That is why many

recent approaches for unsupervised anomaly detection try to

obtain representations of normal data and detect anomalies

by comparing them with test samples’ representations.

Learning representations of normal data in an unsuper-

vised anomaly detection task is closely related to the goal

*These authors contributed equally.

of normalizing ﬂow: the model learns invertible mapping

of a complex distribution of normal samples into a simple

distribution such as the normal distribution. The original

usages of the normalizing ﬂow are density estimation and

generating data efﬁciently using invertible mappings (Dinh,

Sohl-Dickstein, and Bengio 2017; Kingma and Dhariwal

2018), whereas some recent approaches (Rudolph, Wandt,

and Rosenhahn 2021; Yu et al. 2021; Gudovskiy, Ishizaka,

and Kozuka 2022) begin to utilize the normalizing ﬂow to

tackle the unsupervised anomaly detection tasks. In partic-

ular, they have achieved state-of-the-art performance on the

MVTec AD (Bergmann et al. 2019) of the industrial domain.

Despite the success of normalizing ﬂow-based anomaly

detection models, they have a limitation for the unsuper-

vised task: performances are unstable in many cases. Specif-

ically, their test performances often ﬂuctuate while train-

ing the model for a long time (e.g. 200 or 400 epochs).

This drawback will be critical in every practical domain be-

cause one must use the unsupervised trained models that

have been trained sufﬁciently. One possible reason is the

limited expressive power of normalizing ﬂow with the ﬁxed

base (prior) distribution. Current approaches as Rudolph,

Wandt, and Rosenhahn (2021); Yu et al. (2021); Gudovskiy,

Ishizaka, and Kozuka (2022) ﬁx the base distribution of nor-

malizing ﬂow as N(0, I). However, as depicted in Figures 1

(b) and (c), most outputs of the model do not follow N(0, I),

while they utilize the likelihoods in the base distribution as

a score function:

−exp 1

|C|X

c∈C

−zT

czc

2!,(1)

where Cis an index set for output channels. Even the score

function requires an assumption for the theoretical validity

that normal samples are transformed to the base distribution

while training, it fails for the models to learn N(0, I). This

phenomenon occurs pervasively for normalizing ﬂow-based

anomaly detection models.

To tackle this problem by considering the reason

mentioned above, we propose an algorithm called the

Alternating Training method to Update Base distribution

(AltUB) that performs better and learns more stably with

normalizing ﬂow-based anomaly detection models. Our

method introduces alternating updates to learn a base dis-

tribution more actively and to allow the model to adapt to

arXiv:2210.14913v1 [cs.LG] 26 Oct 2022

7500

15000

0.00 0.25 0.50 0.75 1.00

p−value of shapiro−wilk test

count

(a)

50000

100000

150000

0.00 0.25 0.50 0.75 1.00

p−value of Kolmogorov−Smirnov test

count

(b)

0.25

0.50

0 1 2 3 4

Outputs of FastFlow

density

(c)

Figure 1: Histograms of outputs of FastFlow. This ﬁgure shows that outputs of normalizing ﬂow follow an arbitrary normal

distribution, not the base distribution N(0, I). Feature extractor: CaiT, input: MVTec AD capsule category. (a) The result of

the Shapiro-Wilk test (Shapiro and Wilk 1965) applied to values of each spatial location of each channel among outputs from

normal images. The distribution with a p-value >0.05 is usually considered normal. (b) The result of the Kolmogorov-Smirnov

test (KS-test) (Massey 1951) applied to the same ones with (a). p-value >0.05 implies the distribution is highly likely to be

N(0,1). (c) Visualization of one of the distributions. Ideally, it should follow N(0,1). However its distribution is roughly

N(2,0.5).

Flow

AltUB

𝝁𝜮

𝒛

Likelihood

Backward(𝜂1)

Backward(𝜂2)

Figure 2: The training process of AD ﬂow+AltUB. The

shaded (Flow, µ, Σ) are trainable, and the two backward

lines (blue solid line and red dotted line) indicate the al-

ternating updates. While training, the model learns so that

likelihood is high and the anomaly score is low for normal

samples.

the changed base distribution gradually. This simple but ef-

fective updating scheme improves the expressive power of

the normalizing ﬂow, especially for anomaly detection.

To verify the stability of our method under the unsuper-

vised anomaly detection task, we measure the average per-

formance for some training epoch intervals as well as the

best performance. In the experimental results, our method

improves the stability of normalizing ﬂow-based anomaly

detection models. In addition, our method achieves state-

of-the-art performance on anomaly segmentation of both

MVTec AD and BTAD datasets. These results support that

our method is learning-stable and performing better.

Contributions In summary, we propose AltUB for the

anomaly detection tasks with the following contributions.

• We investigate that normalizing ﬂow models for anomaly

detection commonly fail to transform normal images into

N(0, I).

• We suggest the update of the base distribution can solve

the above defect, and propose a proper method to update

the base distribution: Alternating training.

• Our model achieves state-of-the-art performance on

anomaly detection of MVTec AD and BTAD datasets.

2 Related work

2.1 Normalizing ﬂow-based anomaly detection

models

Unsupervised anomaly detection models based on normaliz-

ing ﬂow have shown high performance on various practical

tasks (Rudolph, Wandt, and Rosenhahn 2021; Yu et al. 2021;

Gudovskiy, Ishizaka, and Kozuka 2022). Normalizing ﬂow-

based models ﬁrst obtain representations of normal images

from a pre-trained feature extractor. They utilize the repre-

sentations to learn a distribution of normal data. Because of

their simple yet powerful idea of invertible mappings, they

effectively describe the distribution. To detect anomalies,

they estimate the likelihood of the base distribution of test

samples.

However, their expressive power is not enough as shown

in Figure 1. It might cause of their low stability on perfor-

mance while learning i.e. performance decreases as learning

progresses. In this paper, we suggest a simple method to in-

crease the expressive power of normal data to improve the

AUROC score.

2.2 Methods to update the base distribution of

normalizing ﬂow

Some prior works (Kingma and Dhariwal 2018; Bhat-

tacharyya et al. 2020) have suggested methods to train the

base distribution of normalizing ﬂow for image generation

and density estimation. Glow (Kingma and Dhariwal 2018),

which is the well-known model for image generation, tries to

update the channel-wise normal base distribution through a

single layer of convolution neural network. Also, mAR-SCF

(Bhattacharyya et al. 2020) applies the multi-scale autore-

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AltUB:AlternatingTrainingMethodtoUpdateBaseDistributionofNormalizingFlowforAnomalyDetectionYeongminKim*1,HuiwonJang*2,DongKeonLee3,Ho-JinChoi31SchoolofFreshman,KAIST2DepartmentofMathematicalScience,KAIST3SchoolofComputing,KAIST(cytotoxicity8/huiwoen0516/hagg30/hojinc)@kaist.ac.krAbstractUnsupervised...

展开>> 收起<<

AltUB Alternating Training Method to Update Base Distribution of Normalizing Flow for Anomaly Detection Yeongmin Kim1 Huiwon Jang2 DongKeon Lee3 Ho-Jin Choi3.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

AltUB Alternating Training Method to Update Base Distribution of Normalizing Flow for Anomaly Detection Yeongmin Kim1 Huiwon Jang2 DongKeon Lee3 Ho-Jin Choi3

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: