Data-Driven Computational Imaging for Scientiﬁc Discovery

2025-08-18 3 0 6.56MB 15 页 10玖币

侵权投诉

Data-Driven Computational Imaging for

Scientiﬁc Discovery

Andrew Olsen∗1, Yolanda Hu∗1, Vidya Ganapati1,2

1Swarthmore College, 2Lawrence Berkeley National Laboratory

Abstract

In computational imaging, hardware for signal sampling and software for object

reconstruction are designed in tandem for improved capability. Examples of such

systems include computed tomography (CT), magnetic resonance imaging (MRI),

and superresolution microscopy. In contrast to more traditional cameras, in these

devices, indirect measurements are taken and computational algorithms are used for

reconstruction. This allows for advanced capabilities such as super-resolution or 3-

dimensional imaging, pushing forward the frontier of scientiﬁc discovery. However,

these techniques generally require a large number of measurements, causing low

throughput, motion artifacts, and/or radiation damage, limiting applications. Data-

driven approaches to reducing the number of measurements needed have been

proposed, but they predominately require a ground truth or reference dataset, which

may be impossible to collect. This work outlines a self-supervised approach and

explores the future work that is necessary to make such a technique usable for

real applications. Light-emitting diode (LED) array microscopy, a modality that

allows visualization of transparent objects in two and three dimensions with high

resolution and ﬁeld-of-view, is used as an illustrative example. We release our code

https://github.com/vganapati/LED_PVAE

and our experimental data at

https://doi.org/10.6084/m9.figshare.21232088.

1 Introduction

Computational imaging systems, which reconstruct objects from indirect measurements, are ubiqui-

tous in modern scientiﬁc research. For example, computed tomography allows for 3-dimensional

visualization by collecting x-ray projections through an object, and is used in a range of ﬁelds

including medicine [1], biology [2], materials science [3], and geoscience [4]. Another computational

imaging modality, light-emitting diode (LED) array microscopy (also known as Fourier ptycho-

graphic microscopy), has shown success in quantitative 2-dimensional and 3-dimensional phase

imaging [5–11], with applications in pathology [12, 13] and biology [14, 15].

Though computational imaging methods have achieved a high degree of success in spatial resolution,

the temporal resolution (imaging speed) remains low. Increasing speed remains an active area

of research, as success will allow visualization in unprecedented regimes of spatial and temporal

resolution. In x-ray computed tomography, for example, thousands of 2-dimensional images must

be collected for eventual object reconstruction. In LED array microscopy, the light source of a

conventional wide-ﬁeld microscope is replaced with a 2-dimensional LED array. Each LED of the

array is individually addressable with tunable brightness, allowing different patterns to be illuminated.

Generally, one image is collected per LED, and arrays may consist of hundreds of LEDs.

Data-driven deep learning methods, using a reference training dataset of measurement-reconstruction

pairs, have been widely proposed to improve the temporal resolution of computational imaging.

∗Equal contribution. Correspondence to vidyag@berkeley.edu.

NeurIPS 2022 AI for Science Workshop.

arXiv:2210.16709v1 [eess.IV] 29 Oct 2022

However, the necessity of a training dataset makes the technique prohibitive in many applications of

scientiﬁc discovery. A chicken-and-egg problem arises in the case of fragile or live specimens: without

a reference object dataset, we cannot create a faster imaging method, but without the faster imaging

method the training object dataset cannot be obtained. In this work, we outline a reconstruction

method that only requires a representative dataset of sparse or partial measurements on each object. To

circumvent the need for complete training dataset pairs, we look to jointly reconstruct a set of similar

objects, each with a low number of measurements. By pooling information from measurements

across the set and incorporating the known forward physics of imaging, we aim to jointly infer the

prior distribution and posterior distributions. We aim to allow for improved reconstructions with

fewer measurements per object by using information from multiple similar objects.

More precisely, computational imaging aims to reconstruct some object

from a sequence of

noisy measurements

M= [M1, M2, ..., Mn]

. We aim to lower the total number of measurements

to minimize data acquisition time. We assume that we have a set of

objects

{O1, O2, ..., Om}

sampled from some distribution

P(O)

, and we aim to reconstruct all objects in the set. For each of

the

objects, we are allowed

measurements. Each sequence of measurements for an object

Mj= [Mj1, Mj2, ..., Mjn]

is obtained with chosen hardware parameters

pj= [pj1, pj2, ..., pjn]

(e.g. rotation angles in the case of computed tomography or the LED illumination patterns in the

case of LED array microscopy). We assume that the forward model physics

P(M|O;p) = P(M|O)

is known. For every object

, we aim to ﬁnd the posterior distribution

P(O|M) = P(M|O)P(O)

P(M)

The following problems arise in ﬁnding the posterior: (1) construction of the prior

P(O)

with no

directly observed

, only indirect measurements

on each object of the set, and (2) calculating

P(O|M)

in a tractable manner. To efﬁciently solve this problem, we create a novel technique through

a reformulation of variational autoencoders. The probabilistic formulation considered in this work

permits uncertainty quantiﬁcation, in contrast to most reconstruction algorithms that only yield a

point estimate.

2 Related Work

Deep learning has been widely applied to reduce the data acquisition burden of computational imaging

systems. In one line of research, training pairs of sparse measurements and corresponding high

quality reconstructions are used to train a deep convolutional neural network, and implicitly embed

prior information [16

–

40]. Subsequent sparse measurements can be reconstructed with a forward pass

of the trained neural network, with the beneﬁt of avoiding computationally costly iterative algorithms.

Deep neural network approaches for object reconstruction have the advantage of incorporating

knowledge about prior data and fast inference, but more traditional iterative (model-based) methods

have the advantage of utilizing the known forward physics model (i.e. how measurements are

generated, given the object). The advantages of these two approaches are combined by unrolling

an iterative method, with each iteration forming a layer of a neural network [41

–

53]. This unrolled

deep neural network can be trained to optimize iterative algorithm hyperparameters for a given

training dataset, effectively optimizing an optimizer. The unrolled iterative methods have been shown

to require less training data and time than a convolutional neural network approach, due to the

incorporation of the forward model.

Building off of this literature, a second body of approaches attempts to include the measurement

process in neural network training, to discern the optimal measurement parameters (e.g. the LED

illumination patterns in LED array microscopy) for sparse sampling and subsequent reconstruction.

In these works, high quality reconstructions are needed, and corresponding noisy measurements are

emulated with the known forward physics. The measurement process is included as the encoder part

of an autoencoder neural network, and co-optimized with the reconstruction algorithm, which forms

the decoder. Many works use a convolutional neural network as the decoder [54

–

75] and others

use an unrolled iterative solver [76

–

78]. Co-optimizing the measurement parameters has the beneﬁt

of reducing the measurements required for computational imaging further than keeping them ﬁxed

during training. However, this approach still requires a reference training dataset.

In this work, we look to remove the need for ground-truth or reference reconstructions. We aim to

create a reconstruction method that only requires a representative dataset of sparse measurements

on each object. This task has been previously undertaken, usually with generative adversarial

networks [79

–

87]. The intuition is that by using different experimental measurement parameters

for every object of the set, it is possible to build a general understanding of what an object should

look like (i.e. the prior). However, these methods all lack probabilistic outputs. In this work, we

propose a method based on variational autoencoders that solves for the posterior distribution in

a principled manner and outline some of the progress required to make this technique usable for

scientiﬁc discovery.

3 Physics-Informed Variational Autoencoder

We assume that we have a set of

objects

{O1, O2, ..., Om}

drawn from

P(O)

, where

P(O)

unknown and we cannot directly measure

. For each object

, we are allowed to take

indirect

measurements

M= [M1, M2, ..., Mn]

. The measurement

on an object

is obtained with chosen

hardware parameters

p= [p1, p2, ..., pn]

, and the forward model

P(M|O;p) = P(M|O)

is known

through the physics of image formation. We aim to determine the posterior distribution

P(O|M)

for

every object in the set.

We propose a general framework for posterior estimation that is inspired by the mathematics of the

variational autoencoder [88,89]. In a variational autoencoder, the goal is to learn how to generate

new examples, sampled from the same underlying probability distribution as a training dataset of

objects. To accomplish this task, a latent random variable

is created that describes the space on

a lower-dimensional manifold. A deep neural network deﬁnes a function (the “decoder”) from a

sample of

to a probability distribution

P(O|z)

, see Fig. 1. Deep neural networks are chosen for

function approximation due to their theoretical ability to approximate any function [90

–

92] and

practical success in approximating high-dimensional functions [93]. The parameters of the deep

neural network are optimized to maximize the probability of generating the objects in the training set.

Figure 1: Generating object

from a latent variable

In this work, we aim to ﬁnd the posterior

probability distribution

P(O|M)

, where

is the object being reconstructed and

is the measurement. In our case, we only

have a dataset of noisy measurements

and no ground truth objects

, but a known

forward model,

P(M|O)

. Thus, instead of

maximizing the probability of generating

, we can maximize the probability of generating

, a

formulation we call the “physics-informed variational autoencoder.”

Figure 2: The physics-informed variational autoencoder.

We aim to maximize

P(M) = R R P(M|O)P(O|z)P(z)dOdz

. To compute this integral in a

computationally tractable manner, we can approximate with sampled values. However, for most

values of

and

, the probability

P(M|O, z)

is close to zero, causing poor scaling of sampled

estimates. Similar to a variational autoencoder, our framework solves this problem by estimating the

parameters of

P(z|M)

by processing the measurements

using a function with trainable parameters

(called the “encoder,” usually a deep neural network). The estimate of

P(z|M)

is denoted

Q(z|M)

The Kullback–Leibler divergence between the distributions is given by

D[Q(z|M)||P(z|M)] =

Ez∼Q[log Q(z|M)−log P(z|M)]

. We also have, by Bayes’ Theorem,

log P(z|M) = log P(M|z)+

log P(z)−log P(M). Combining the expressions yields:

log P(M)−D[Q(z|M)||P(z|M)] = Ez∼QZP(M|O)P(O|z)dO−D[log Q(z|M)|| log P(z)] .

The ﬁrst term on the right side of this expression can be estimated with sampled values. As

Kullback–Leibler divergence is always

≥0

and reaches

when

Q(z|M) = P(z|M)

, maximizing

the right side (deﬁned here as the loss) during training causes

P(M)

to be maximized while forcing

Q(z|M)

towards

P(z|M)

. In contrast to a conventional variational autoencoder, we do not attempt

to use this formulation to synthesize arbitrary objects

by sampling

P(z)

directly. This framework

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Data-DrivenComputationalImagingforScienticDiscoveryAndrewOlsen1,YolandaHu1,VidyaGanapati1;21SwarthmoreCollege,2LawrenceBerkeleyNationalLaboratoryAbstractIncomputationalimaging,hardwareforsignalsamplingandsoftwareforobjectreconstructionaredesignedintandemforimprovedcapability.Examplesofsuchsystems...

展开>> 收起<<

Data-Driven Computational Imaging for Scientiﬁc Discovery.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Data-Driven Computational Imaging for Scientiﬁc Discovery

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: