SINCO A Novel structural regularizer for image compression using implicit neural representations Harry Gao Weijie Gan Zhixin Sun and Ulugbek S. Kamilov

2025-05-03 0 0 1.92MB 7 页 10玖币

侵权投诉

SINCO: A Novel structural regularizer for image compression

using implicit neural representations

Harry Gao∗, Weijie Gan∗, Zhixin Sun, and Ulugbek S. Kamilov

∗These authors contributed equally to this work.

Computational Imaging Group, Washington University in St. Louis, MO 63130, USA

{harrygao, weijie.gan, zhixin.sun, kamilov}@wustl.edu

Abstract

Implicit neural representations (INR) have been recently proposed as deep learning (DL) based solutions for image

compression. An image can be compressed by training an INR model with fewer weights than the number of image

pixels to map the coordinates of the image to corresponding pixel values. While traditional training approaches for

INRs are based on enforcing pixel-wise image consistency, we propose to further improve image quality by using a

new structural regularizer. We present structural regularizationfor INR compression (SINCO) as a novel INR method

for image compression. SINCO imposes structural consistency of the compressed images to the groundtruth by using

a segmentation network to penalize the discrepancy of segmentation masks predicted from compressed images. We

validate SINCO on brain MRI images by showing that it can achieve better performance than some recent INR

methods.

1 Introduction

Image compression is an important step for enabling efﬁcient transmission and storage of images in many applica-

tions. It is widely used in biomedical imaging due to the high-dimensional nature of data. While traditional image

compression methods are based on ﬁxed image transforms [1,2], deep learning (DL) has recently emerged as a power-

ful data-driven alternative. The majority of DL-based compression methods are based on training autoencoders to be

invertible mappings from image pixels to quantized latent representations [3–5].

In this work, we seek an alternative to the autoencoder-based compression methods by focusing on a recent paradigm

using implicit neural representations (INRs). INR refers to a class of DL techniques that seek to learn a mapping

from input coordinates (e.g., (x, y)) to the corresponding physical quantities (e.g., density at (x, y)) by using a multi-

layer perceptron (MLP) [6–10]. Recent studies have shown the potential of INR in image compression [11–15]. The

key idea behind INR based compression is to train a MLP to represent an image and consider the weights of the

trained model as the compressed data. One can then reconstruct the image by evaluating the pre-trained MLP on the

desired pixel locations. The traditional training strategy for image compression using INRs seeks to enforce image

consistency between predicted and groundtruth image pixels. On the other hand, it is well-known that image quality

can be improved by infusing prior knowledge on the desired images [16,17]. Based on this observation, we propose

Structural regularIzatioNfor INR COmpression (SINCO) as new method for improving INR-based image compression

using a new structural regularizer. Our structural regularizer seeks to improve the Dice score between the groundtruth

segmentation maps and those obtained from the INR compressed image using a pre-trained segmentation network.

We validate SINCO on brain MR images by showing that it can lead to signiﬁcant improvements over the traditional

INR-based image compression methods. We show that the combination of the traditional image-consistency loss and

our structural regularizer enables SINCO to learn an INR that can better preserve desired image features.

arXiv:2210.14974v1 [eess.IV] 26 Oct 2022

1 2 4 8

image coordinate

positional encoding

segmentation network

raw image

compressed image

groundtruth segmentaiton

predicted segmentaiton

back-propagation no-update

SIREN

-2.4

-2 -1.6 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 1.6 2

2.4

-1.2

-0.8

-0.4

0.4

0.8

1.2

sine function

NeRF

linear layer

linear layer + ReLU

Figure 1: SINCO consists of two components: (a) a multi-layer perceptron (MLP) for compressing an image by

mapping its coordinates to the corresponding pixel values; (b) a segmentation network that predicts segmentation

masks from the compressed image. Unlike traditional INR methods that only enforce consistency between compressed

and groundtruth images, SINCO uses information from a regularizer that penalizes the discrepancy between predicted

and groundtruth segmentation masks.

2 Background

INR (also referred to as neural ﬁelds) denotes a class of algorithms for continuously representing physical quanti-

ties using coordinate-based MLPs (see a recent review [6]). Recent work has shown the potential of INRs in many

imaging and vision tasks, including novel view synthesis in 3D rendering [7], video frame interpolation [8], computed

tomography [9] and dynamic imaging [10]. The key idea behind INR is to train a MLP to map spatial coordinates

to corresponding observed physical quantities. After training, one can evaluate the pre-trained MLP on desired coor-

dinates to predict the corresponding physical quantities, including on locations that were not part of training. Let c

denotes a vector of input coordinates, vthe corresponding physical quantity, and Mθa MLP with trainable parameter

θ∈Rn. The INR training can be formulated as

θ= arg min

θ∈Rn

i=1

`inr(Mθ(ci), vi).(1)

where N≥1denotes the number training pairs (c, v). The common choices for `inr include `2and `1norms.

INRs have been recently used for image compression [11–14] (see also a recent evaluation in medical imaging [15]).

COmpressed Implicit Neural representations (COIN) [11] is a pioneering work based on training a MLP by mapping

the pixel locations (i.e., c= (x, y)) of an image to the pixel values. The pre-trained MLP in COIN is quantized and

then used as the compressed data. In order to reconstruct the image, one can evaluate the model on the same pixel

locations used for training. Several papers have investigated the meta-learning approach to accelerate COIN by ﬁrst

training a MLP over a large collection of datapoints and then ﬁne-tuning it on an instance-dependent one [12,13,18].

Two recent papers proposed to regularize INR-based image compression by using `0- and `1-norm penalties on the

weights of the MLP to improve the compression rate [12,14].

The structural regularization in SINCO is based on image segmentation using a pre-trained convolutional neural

network (CNN) (see a comprehensive review of the topic [19]). There exists a rich body of literature in the context

of DL-based image segmentation that can be integrated into SINCO [20–22]. To the best of our knowledge, no prior

work has considered higher-level structural regularization in the context of INR-based image compression. It is worth

mentioning that our structural regularizer is fully compatible to the existing INR compression methods; for example,

one can easily combine our structural regularizer with an additional `0-regularizer.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

SINCO:ANovelstructuralregularizerforimagecompressionusingimplicitneuralrepresentationsHarryGao,WeijieGan,ZhixinSun,andUlugbekS.KamilovTheseauthorscontributedequallytothiswork.ComputationalImagingGroup,WashingtonUniversityinSt.Louis,MO63130,USAfharrygao,weijie.gan,zhixin.sun,kamilovg@wustl.eduAbst...

展开>> 收起<<

SINCO A Novel structural regularizer for image compression using implicit neural representations Harry Gao Weijie Gan Zhixin Sun and Ulugbek S. Kamilov.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

SINCO A Novel structural regularizer for image compression using implicit neural representations Harry Gao Weijie Gan Zhixin Sun and Ulugbek S. Kamilov

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: