DA-VSR Domain Adaptable Volumetric Super-Resolution For Medical Images Cheng Peng1 S. Kevin Zhou23 and Rama Chellappa1

2025-05-06 0 0 611.03KB 11 页 10玖币

侵权投诉

DA-VSR: Domain Adaptable Volumetric

Super-Resolution For Medical Images

Cheng Peng1, S. Kevin Zhou2,3, and Rama Chellappa1

1Johns Hopkins University, MD, USA

2Medical Imaging, Robotics, and Analytic Computing Laboratory and Engineering

(MIRACLE) Center, School of Biomedical Engineering & Suzhou Institute for

Advance Research, University of Science and Technology of China, Suzhou, China

3Key Lab of Intelligent Information Processing of Chinese Academy of Sciences

(CAS), Institute of Computing Technology, CAS, Beijing, China

cpeng26@jhu.edu, s.kevin.zhou@gmail.com, rchella4@jhu.edu

Abstract. Medical image super-resolution (SR) is an active research

area that has many potential applications, including reducing scan time,

bettering visual understanding, increasing robustness in downstream tasks,

etc. However, applying deep-learning-based SR approaches for clinical

applications often encounters issues of domain inconsistency, as the test

data may be acquired by diﬀerent machines or on diﬀerent organs. In this

work, we present a novel algorithm called domain adaptable volumetric

super-resolution (DA-VSR) to better bridge the domain inconsistency

gap. DA-VSR uses a uniﬁed feature extraction backbone and a series

of network heads to improve image quality over diﬀerent planes. Fur-

thermore, DA-VSR leverages the in-plane and through-plane resolution

diﬀerences on the test data to achieve a self-learned domain adaptation.

As such, DA-VSR combines the advantages of a strong feature generator

learned through supervised training and the ability to tune to the id-

iosyncrasies of the test volumes through unsupervised learning. Through

experiments, we demonstrate that DA-VSR signiﬁcantly improves super-

resolution quality across numerous datasets of diﬀerent domains, thereby

taking a further step toward real clinical applications.

1 Introduction

Medical imaging such as Magnetic Resonance Imaging (MRI) and Computed

Tomography (CT) are crucial to clinical diagnosis. To facilitate faster and less

costly acquisitions, it is routine to acquire a few high-resolution cross sectional

images in CT/MRI, leading to a low through-plane resolution when the acquired

images are organized into an anisotropic volume. The anisotropic volumes lead

to diﬃculties in understanding the patient’s anatomy both for physicians and au-

tomated algorithms [12,17]. One way to address this is through super-resolution

(SR) algorithms [26,27], which upsample along the axis with a low resolution. SR

has witnessed great improvement in the image domain with Convolutional Neu-

ral Network (CNN)-based algorithms [4,8,9,20], where the formulation typically

involves supervised learning between a low resolution (LR) image and its paired

arXiv:2210.05117v1 [eess.IV] 11 Oct 2022

2 Cheng Peng, S. Kevin Zhou, and Rama Chellappa

high resolution (HR) groundtruth. Various improvements have been made to

reduce computation [5,14], enhance feature extraction eﬃciency [10,22,23], and

improve robustness [15,21].

Volumetric SR for medical images poses unique challenges. Firstly, the high

dimensional and anisotropic nature of volumetric images lead to diﬃculties in

computational cost and learning eﬃciency. While there exists many work on 2D

medical image SR [18,11,25,3,19,6], few directly tackle 3D medical image SR

due to high computational cost and limited sample size. Chen et al. [2] apply a

DenseNet-based CNN algorithms called mDCSRN on volumetric data with 3D

kernels. Wang et al. [17] ease the 3DCNN memory bottleneck by using a more

eﬃciently implemented DenseNet with residual connections. These methods still

require patch-by-patch inference on large-size volume, which can lead to unde-

sirable border artifacts and ineﬃciency. Peng et al. [13] propose SAINT, which

can super-resolve volumetric images with multiple upsampling factors with a

single network. Furthermore, it addresses the memory constraint at inference by

performing an ensemble of 2D SR operations on through-plane images.

Another challenge arises from the need for high reliability. Under a supervised

learning framework, if a test image comes from a distribution not well represented

in training, e.g. of a diﬀerent body part or by a diﬀerent machine, performance

often degrades in unexpected ways. Therefore, semi-supervised or self-supervised

SR methods provide distinct advantages if they can learn directly from test

datasets. Zhao et al. [24] propose SMORE, a self-supervised SR algorithm that

leverages the high in-plane resolution to create LR-HR pairs for learning, and

applies the learned model on lower through-plane resolution. Implicitly, SMORE

assumes that the in-plane and through-plane images are from same or similar

distributions, which may not be true for many cases.

To address the issue of robustly super-resolving volumetric medical images,

we propose a novel algorithm named Domain Adaptable Volumetric Super-

Resolution (DA-VSR). DA-VSR follows SAINT’s thinking in addressing vol-

umetric SR based on a series of slice-wise SR. DA-VSR uses a single feature

extraction backbone and assigns small, task-speciﬁc network heads for upsam-

pling and fusion. Inspired by SMORE, DA-VSR leverages the resolution diﬀer-

ences across dimensions as a self-supervised signal for domain adaptation at test

time. Speciﬁcally, DA-VSR designs an additional self-supervised network head

that can help align features on test images through in-plane super-resolution.

As a result, DA-VSR enjoys the beneﬁt of a strong feature backbone obtained

by supervised training, and the ability to adapt to various distributions through

unsupervised training. To summarize,

–We design a slice-based volumetric SR network called DA-VSR. DA-VSR

uses a Uniﬁed Feature Extraction (UFE) backbone and a series of lightweight

network heads to perform super-resolution.

–We propose an in-plane SR head that propagates gradients to the UFE

backbone both in training and testing. As such, DA-VSR is more robust,

and can adapt its features to the test data distribution.

Title Suppressed Due to Excessive Length 3

–We experiment with a diverse set of medical imaging data on diﬀerent parts

of the organ, and ﬁnd large quantitative and visual improvement in SR

quality on datasets out of the training distribution.

2 Domain Adaptable Volumetric Super-Resolution

Fig. 1: The overall pipeline of Domain Adaptable Volumetric Super-Resolution

(DA-VSR). DA-VSR contains three stages. The network parameters are ﬁrst

trained in supervised setting. An additional adaptation stage is proposed to ﬁt

to test data. Finally inference is done through an adapted feature backbone and

network heads. Networks of the same color share weights.

Consider I(x, y, z)∈RX×Y×Zas a densely sampled volumetric medical im-

age. Following the notations in [13], we refer to x,y, and zas the sagittal, coronal,

and axial axis, and Ix(y, z), Iy(x, z), and Iz(x, y) as the sagittal, coronal, and

axial slices. The task of super-resolution seeks to recover I(x, y, z) from its par-

tially observed, downsampled version I↓(x, y, z). This work focuses on ﬁnding a

transformation F:RX×Y×Z

rz→RX×Y×Zthat super-resolves an axially sparse

volume I↓rz(x, y, z) to I(x, y, z), where rzis the sparsity factor in the axial axis.

The super-resolving function Fis most popularly approximated through a CNN

FθS, where θSdenotes the network parameters learned from a training set Sthat

contains LR-HR pairs {IS

↓rz(x, y, z), IS(x, y, z)}for supervised learning. While

FθSmay be near-optimal for S, its performance degrades when used on a test set

Tfrom a diﬀerent distribution. Hence, we would like to approximate a better

FθTbased on θSand IT

↓rz(x, y, z). As shown in Fig. 1, DA-VSR consists of a

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DA-VSR:DomainAdaptableVolumetricSuper-ResolutionForMedicalImagesChengPeng1,S.KevinZhou2;3,andRamaChellappa11JohnsHopkinsUniversity,MD,USA2MedicalImaging,Robotics,andAnalyticComputingLaboratoryandEngineering(MIRACLE)Center,SchoolofBiomedicalEngineering&SuzhouInstituteforAdvanceResearch,UniversityofSc...

展开>> 收起<<

DA-VSR Domain Adaptable Volumetric Super-Resolution For Medical Images Cheng Peng1 S. Kevin Zhou23 and Rama Chellappa1.pdf

共11页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

DA-VSR Domain Adaptable Volumetric Super-Resolution For Medical Images Cheng Peng1 S. Kevin Zhou23 and Rama Chellappa1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: