Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images

2025-05-03 0 0 1.26MB 16 页 10玖币

侵权投诉

Medical Image Analysis (2023)

Contents lists available at ScienceDirect

Medical Image Analysis

journal homepage: www.elsevier.com/locate/media

Dual-distribution discrepancy with self-supervised reﬁnement for anomaly detection in

medical images

Yu Caia, Hao Chenb,c,∗, Xin Yangd,YuZhoud, Kwang-Ting Chenga,b

aDepartment of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China

bDepartment of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China

cDepartment of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong, China

dSchool of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China

ARTICLE INFO

Article history:

Received 22 August 2022

Received in ﬁnal form 13 February 2023

Accepted 6 March 2023

Available online 13 March 2023

Keywords:

Anomaly detection

Reconstruction networks

Self-supervised learning

Benchmark

ABSTRACT

Medical anomaly detection is a crucial yet challenging task aimed at recognizing ab-

normal images to assist in diagnosis. Due to the high-cost annotations of abnormal

images, most methods utilize only known normal images during training and identify

samples deviating from the normal proﬁle as anomalies in the testing phase. Many

readily available unlabeled images containing anomalies are thus ignored in the train-

ing phase, restricting the performance. To solve this problem, we introduce one-class

semi-supervised learning (OC-SSL) to utilize known normal and unlabeled images for

training, and propose Dual-distribution Discrepancy for Anomaly Detection (DDAD)

based on this setting. Ensembles of reconstruction networks are designed to model

the distribution of normal images and the distribution of both normal and unlabeled

images, deriving the normative distribution module (NDM) and unknown distribution

module (UDM). Subsequently, the intra-discrepancy of NDM and inter-discrepancy be-

tween the two modules are designed as anomaly scores. Furthermore, we propose a

new perspective on self-supervised learning, which is designed to reﬁne the anomaly

scores rather than directly detect anomalies. Five medical datasets, including chest

X-rays, brain MRIs and retinal fundus images, are organized as benchmarks for eval-

uation. Experiments on these benchmarks comprehensively compare a wide range of

anomaly detection methods and demonstrate that our method achieves signiﬁcant gains

and outperforms the state-of-the-art. Code and organized benchmarks are available at

https://github.com/caiyu6666/DDAD-ASR.

1. Introduction

Medical imaging is of vital importance to the diagnosis of

a wide variety of pathologies. Take the case of chest X-rays

(CXRs), which are the most commonly performed radiological

exam (C¸ allı et al.,2021), widely applied for the diagnosis of

∗Corresponding author.

e-mail: jhc@cse.ust.hk (Hao Chen)

tens of lung diseases such as pneumonia, nodules, lung opac-

ity, pneumothorax, etc. To alleviate the burden on radiologists

in reading CXRs and improve diagnosis eﬃciency, automatic

CXR analysis using deep learning is becoming popular (Luo

et al.,2020,2021,2022a,b). However, such methods require

the annotation of images, which is diﬃcult, relies on the experi-

ence of professional experts, and is time-consuming and labor-

intensive. This has motivated the development of intelligent

systems to help radiologists automatically detect and localize

potential abnormalities using few or even no annotations.

arXiv:2210.04227v3 [cs.CV] 18 Mar 2023

2 Yu Cai et al. /Medical Image Analysis (2023)

Normal image Unlabeled image

(b) One-Class Classification (c) The proposed mode

Abnormal image

(a) Semi-supervised

Normal image Unlabeled image

(b) One-Class Classification (c) The proposed mode

Abnormal image

(a) Semi-supervised

(a) One-class

Normal image Unlabeled image

(b) One-Class Classification (c) The proposed mode

Abnormal image

(a) Semi-supervised

(b) Semi-supervised

Normal image Unlabeled image

(b) One-Class Classification (c) The proposed mode

Abnormal image

(a) Semi-supervised

Fig. 1. Diﬀerent training modes for medical anomaly detection. (a) One-

class classiﬁcation mode, utilizing only normal images, is the most popular,

but wastes unlabeled images. (b) Semi-supervised mode requires labeled

normal and abnormal images, and mostly normal unlabeled images, thus

infeasible in clinical practice. (c) The introduced OC-SSL mode utilizes

normal and unlabeled images with arbitrary anomaly rates.

As annotations of normal images from healthy subjects are

relatively easy to obtain while those of anomalies are complex,

various and usually diﬃcult to collect, most existing methods

consider anomaly detection as a one-class classiﬁcation (OCC)

problem (Ruﬀet al.,2018), where only normal images are uti-

lized for training and samples not conforming to the normal

proﬁle are identiﬁed as anomalies in the testing phase; thus

there is no need for annotation of abnormal images during train-

ing. This setting has been extensively studied in anomaly de-

tection for both computer vision tasks (Ruﬀet al.,2021) and

medical image analysis (Baur et al.,2021). Nevertheless, due

to the lack of training on real abnormal images, the discrim-

inative capability of these methods is limited. Meanwhile, in

medical images analysis, an important fact is ignored that, dif-

ferent from the application scenarios in computer vision tasks,

like industrial defect detection (Bergmann et al.,2019) and

video anomaly detection (Sultani et al.,2018;Li et al.,2013),

where abnormal cases are rare, medical clinical practice pro-

vides plenty of readily available unlabeled images with a cer-

tain anomaly rate (AR). These unlabeled images, containing

rich anomalous features, are wasted by methods based on the

OCC setting, which restricts the performance of anomaly de-

tection.

Although some works have explored the utilization of unla-

beled samples, the unlabeled abnormal samples have yet to be

exploited successfully. Deep SAD (Ruﬀet al.,2019) introduced

semi-supervised anomaly detection, however, it works under

the condition that both labeled normal and abnormal samples

are available, while the unlabeled data is mostly normal. This

condition is diﬃcult to achieve in practice, while anomalies in

unlabeled data are not exploited. One-class SVM (OC-SVM)

(Sch¨

olkopf et al.,1999) and Support Vector Data Description

(SVDD) (Tax and Duin,2004) utilize nonzero slack variables to

penalize the objective function and learn soft margins, and thus

tolerate a small number of outliers in the training set. However,

they essentially try to reduce the eﬀects of unlabeled abnormal

samples for training on normal data similar to the OCC set-

ting, rather than capture useful information from the unlabeled

abnormal samples. It has been demonstrated that their perfor-

mance will decrease consistently as the abnormal samples in the

unlabeled data increase (Yoon et al.,2022). Up to now, there is

still no notable work leveraging unlabeled images for anomaly

detection eﬀectively.

Anomaly Detection

(e.g., DDAD) Anomaly Score

Refinement

Fixed

Segmentation

(a) Standard self-supervised anomaly detection

Anomaly Detection

(e.g., DDAD) Anomaly Score

Refinement

Fixed

Segmentation

(b) The proposed self-supervised anomaly score reﬁnement

Fig. 2. Comparison of (a) the standard self-supervised anomaly detection

and (b) the proposed self-supervised anomaly score reﬁnement. (a) trains

the network to directly detect the synthetic abnormal patterns from the

input image, while (b) learns to reﬁne the original anomaly score maps for

the ﬁnal accurate abnormal regions.

Therefore, a question is naturally raised: can unlabeled im-

ages provide eﬀective information of abnormalities as a com-

plement to normal images to improve the performance of

anomaly detection? Motivated by this question, in this work,

we introduce and explore one-class semi-supervised learning

(OC-SSL) to train the model on known normal and unlabeled

images. A comparison of the OC-SSL with existing settings is

shown in Fig. 1. As mentioned above, the OCC mode (Fig. 1(a))

has been extensively studied in most existing anomaly detection

works, but plenty of unlabeled images are ignored. Existing

semi-supervised anomaly detection methods (Fig. 1(b)) (Ruﬀ

et al.,2019) require both labeled normal and abnormal sam-

ples, while the unlabeled data should be mostly normal. It is in-

tractable in practice, while unlabeled abnormal samples are not

exploited. The introduced OC-SSL mode (Fig. 1(c)) is capable

of utilizing normal and unlabeled images with arbitrary ARs,

while there is no need for labeled abnormal images. Therefore,

the OC-SSL is more reasonable and consistent with the medical

clinical practice.

Based on the OC-SSL mode, we propose Dual-distribution

Discrepancy for Anomaly Detection (DDAD), as shown in

Fig. 3. To capture information from both known normal im-

ages and unlabeled images, we utilize ensembles of reconstruc-

tion networks to model the distribution of normal images and

the distribution of both normal and unlabeled images, deriving

the normative distribution module (NDM) and unknown dis-

tribution module (UDM). Subsequently, the intra-discrepancy

of NDM and inter-discrepancy between the two modules are

designed as anomaly scores (ASs). To further reﬁne the two

ASs, we design an Anomaly Score Reﬁnement Net (ASR-Net),

which is trained via self-supervised learning. Fig. 2depicts our

comparison with the standard self-supervised anomaly detec-

tion. Instead of learning to directly detect the synthetic abnor-

mal patterns, the proposed ASR-Net learns to map the origi-

nal AS to the ﬁnal accurate abnormal regions, thereby avoid-

ing overﬁtting and leading to better performance. Considering

the lack of publicly available benchmarks for medical anomaly

detection, we for the ﬁrst time collect and organize ﬁve med-

ical datasets including CXRs, brain MRIs and retinal fundus

Yu Cai et al. /Medical Image Analysis (2023) 3

UDM

…

 

… 

NDM

…

 

Stage 1 Stage 2







heat map



…

… 

…







Fig. 3. Overview of the proposed DDAD. In the Stage 1, NDM and UDM model the distribution of known normal images and the distribution of known

normal and unlabeled images, respectively. Then the intra-discrepancy inside NDM and inter-discrepancy between the two modules are designed as

anomaly scores. In the Stage 2, the two anomaly scores are reﬁned and fused by the ASR-Net F(·), deriving the ﬁnal prediction Rdual.

images for evaluation and release them to facilitate other re-

searchers evaluating their methods fairly. Experiments on these

ﬁve datasets demonstrate that the proposed DDAD outperforms

existing state-of-the-art methods, even if without unlabeled im-

ages, while unlabeled images can be utilized to further im-

prove our performance by a large margin. Evaluation on un-

seen diseases further demonstrates the potential of our method

for recognition of rare diseases, whose samples are inaccessi-

ble in the unlabeled data. A comprehensive comparison of a

wide range of anomaly detection methods is also provided on

the ﬁve datasets, revealing the performance of diﬀerent families

of methods and potential trends.

Our main contributions are summarized as follows:

•One-class semi-supervised learning (OC-SSL) is intro-

duced. It utilizes known normal and unlabeled images

with arbitrary ARs for anomaly detection, and is reason-

able and consistent with clinical practice.

•Based on the OC-SSL setting, ensembles of reconstruc-

tion networks are used to model the distribution of train-

ing data in an unsupervised fashion. Speciﬁcally, the NDM

and UDM are designed to model the distribution of known

normal images and the distribution of known normal and

unlabeled images, respectively. It is the ﬁrst time that un-

labeled images are utilized to improve the performance of

anomaly detection.

•Two novel and powerful ASs, the intra-discrepancy inside

NDM and inter-discrepancy between the NDM and UDM,

are proposed to indicate anomalies.

•An Anomaly Score Reﬁnement Net (ASR-Net), trained

via self-supervised learning, is proposed to reﬁne and

fuse the two ASs. Diﬀerent from existing self-supervised

anomaly detection methods that learn to detect synthetic

abnormal patterns, it provides a new perspective on self-

supervised learning, i.e., learning to map the original AS

to the ﬁnal accurate abnormal regions. It avoids overﬁtting

and achieves better performance.

•Five medical datasets that include three modalities are col-

lected and organized, and released as benchmarks for med-

ical anomaly detection. These facilitate a fair comparison

with other methods as there are few related existing bench-

marks.

•Extensive experiments on the ﬁve medical datasets demon-

strate that the proposed method achieves consistent, sig-

niﬁcant gains and outperforms state-of-the-art methods in

anomaly detection. A comprehensive comparison of a

wide range of anomaly detection methods is provided to

reveal the performance of diﬀerent families of methods

and potential trends.

A preliminary version of this work was early accepted for

MICCAI 2022 (Cai et al.,2022). In this paper, the major ex-

tensions include designing a new module, namely ASR-Net,

that provides a new perspective on self-supervised learning in

anomaly detection and improves the performance and robust-

ness signiﬁcantly; adding much more experiments on more

datasets containing diﬀerent modalities; elaborating the anal-

ysis; and providing a more comprehensive literature review.

The rest of this paper is organized as follows: Section 2

presents related works. Section 3describes in detail the pro-

posed DDAD methods with our ASR-Net. In Section 4, exten-

sive experiments on ﬁve datasets are conducted to demonstrate

the eﬀectiveness of our proposed method. Section 5discusses

advantages and limitations of the proposed method, and ana-

lyzes a wide variety of methods to reveal future directions and

trends. We conclude our work in Section 6.

2. Related works

Anomaly detection aims at ﬁnding patterns in data that do

not conform to expected behavior (Chandola et al.,2009). It is

a promising ﬁeld that has been widely applied in a variety of

domains. Due to the diﬃculty of collecting abundant annotated

abnormal samples, almost all the existing works utilize only

4 Yu Cai et al. /Medical Image Analysis (2023)

normal images during training, which is the well-known OCC

setting (Ruﬀet al.,2018).

Classical anomaly detection methods, OC-SVM (Sch¨

olkopf

et al.,1999) and SVDD (Tax and Duin,2004), often fail in high-

dimensional data due to bad computational scalability and the

curse of dimensionality. Their derived Deep SVDD (Ruﬀet al.,

2018) utilizes neural networks to constrain the normal sam-

ples in a hypersphere with minimum volume, handling high-

dimensional data better but suﬀering from the mode collapse.

Most recent state-of-the-art anomaly detection methods focus

on reconstruction and self-supervised learning. As techniques

highly related to our work, ensemble-based uncertainty esti-

mates and semi-supervised learning for anomaly detection are

also described in this section.

2.1. Reconstruction-based Anomaly Detection

Reconstruction-based methods are one of the most popular

families in anomaly detection, especially for medical images

(Baur et al.,2021). They usually utilize generative models, such

as generative adversarial networks (GANs) (Goodfellow et al.,

2014), auto-encoders (AEs) or their variants, to learn a map-

ping function to reconstruct normal images, while the unseen

abnormal images are assumed unable to be reconstructed well

by these models trained with only normal images, and in turn

yield high reconstruction error.

Schlegl et al. (2017) are the ﬁrst to use GANs for anomaly

detection. They proposed AnoGAN to learn the manifold of

normal images. For a query image, a latent feature is found

via an iterative process to generate an image most similar to

the query image. The query image will be identiﬁed as abnor-

mal if there is a large diﬀerence with the best generated image.

To replace the time-consuming iterative process in the testing

phase, Schlegl et al. (2019) further utilized an encoder to learn

the mapping from the retinal OCT image to the latent space, and

derived a fast version of AnoGAN, named f-AnoGAN. How-

ever, these GAN-based methods could suﬀer from memoriza-

tion pitfalls, causing reconstructions to diﬀer anatomically from

the actual input.

Various approaches also used variants of AEs for anomaly

detection, including Variational AE (VAE) (Zimmerer et al.,

2018), Adversarial AE (AAE) (Chen and Konukoglu,2018),

Vector Quantized VAE (VQ-VAE) (Marimont and Tarroni,

2021), etc. To avoid abnormal images being well reconstructed,

Gong et al. (2019) proposed to augment the AE with a mem-

ory module, which can store the latent features of normal train-

ing samples. The reconstruction is obtained from a few most

relevant memory records, thus tending to be close to a normal

image and enlarging the reconstruction errors of abnormal im-

ages. Compared with GAN-based methods, AE-based methods

can preserve more anatomical coherence, but usually generate

blurry reconstructions (Baur et al.,2021), leading to false posi-

tive detection around high-frequency regions (e.g., boundaries).

To mitigate this problem, Mao et al. (2020) proposed to auto-

matically estimate the pixel-level uncertainty of reconstruction

using an AE, which is used to normalize the reconstruction error

and suppress the false positive detection in CXRs signiﬁcantly.

Recently, incorporating adversarial training into AEs has be-

come popular, as it combines the advantages of both. Baur et al.

(2018) demonstrated that AEs with spatial bottlenecks can re-

construct important ﬁne details better than those with dense bot-

tlenecks, and combined the spatial VAE with GAN to improve

the realism of reconstructed normal samples for anomaly detec-

tion in brain MRIs. In addition to adversarial training, Akcay

et al. (2018) used an extra encoder to map the reconstructed

image to the latent space again, and minimized reconstruction

errors in both the image space and latent space during training

to aid in learning the data distribution for the normal samples.

Zaheer et al. (2020) proposed to transform the fundamental role

of a discriminator from identifying real and fake data to distin-

guishing between good and bad quality reconstructions, which

is highly desirable in anomaly detection as a trained AE would

not produce as good reconstructions for abnormal images as

they would for normal images conforming to the learned repre-

sentations.

2.2. Self-Supervised Learning-based Anomaly Detection

Self-supervised learning (Jing and Tian,2020), referring to

learning methods in which networks are explicitly trained us-

ing pretext tasks with generated pseudo labels, has also been

extensively studied for anomaly detection. Sohn et al. (2020)

proposed to ﬁrst learn self-supervised representations from one-

class data and then build one-class classiﬁers on learned repre-

sentations. Based on their proposed framework, they applied

distribution augmentation (Jun et al.,2020) for one-class con-

trastive learning to reduce the uniformity of representations.

Further, Tian et al. (2021) combined distribution-augmented

contrastive learning (Sohn et al.,2020), augmentation predic-

tion (Golan and El-Yaniv,2018), and position prediction (Do-

ersch et al.,2015) to learn feature representations for anomaly-

sensitive detection models. Moreover, Li et al. (2021) proposed

to learn representations by classifying normal data from their

designed CutPaste, and then build a Gaussian density estimator

on learned representations.

In addition to the aforementioned representation-based meth-

ods, some works (Tan et al.,2020,2021;Schl¨

uter et al.,2022)

proposed to manually synthesize defects to train models to de-

tect irregularities. Various image processing approaches have

been designed to synthesize abnormal images, including Cut-

Paste (Li et al.,2021), Foreign Patch Interpolation (FPI) (Tan

et al.,2020), Poisson Image Interpolation (PII) (Tan et al.,

2021), etc. Recently, Schl¨

uter et al. (2022) integrated Pois-

son image editing with rescaling, shifting and a new Gamma-

distribution-based patch shape sampling strategy to synthesize

natural and diverse anomalies. Background constraints and

pixel-level labels derived from the resulting diﬀerence to the

normal image were designed to make the results more relevant

to the task. However, these methods may not generalize well

due to the inherent reliance on the similarity between synthetic

abnormal patterns and real anomalies.

Also, Zavrtanik et al. (2021) proposed to combine the re-

construction network with a self-supervised network. It feeds

the concatenation of the original image and reconstruction re-

sult to a segmentation network trained via self-supervised learn-

ing, which is expected to learn a distance function between the

original and reconstructed anomaly appearance. However, the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MedicalImageAnalysis(2023)ContentslistsavailableatScienceDirectMedicalImageAnalysisjournalhomepage:www.elsevier.com/locate/mediaDual-distributiondiscrepancywithself-supervisedrenementforanomalydetectioninmedicalimagesYuCaia,HaoChenb,c,,XinYangd,YuZhoud,Kwang-TingChenga,baDepartmentofElectronicandC...

展开>> 收起<<

Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images.pdf

共16页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: