Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

2025-05-02 0 0 8.32MB 23 页 10玖币

侵权投诉

Maximum Likelihood Learning of Unnormalized Models

for Simulation-Based Inference

Pierre Glaser 1Michael Arbel 2Samo Hromadka 1Arnaud Doucet 3Arthur Gretton 1

Abstract

We introduce two synthetic likelihood methods

for Simulation-Based Inference (SBI), to conduct

either amortized or targeted inference from exper-

imental observations when a high-ﬁdelity simula-

tor is available. Both methods learn a conditional

energy-based model (EBM) of the likelihood us-

ing synthetic data generated by the simulator, con-

ditioned on parameters drawn from a proposal

distribution. The learned likelihood can then be

combined with any prior to obtain a posterior es-

timate, from which samples can be drawn using

MCMC. Our methods uniquely combine a ﬂex-

ible Energy-Based Model and the minimization

of a KL loss: this is in contrast to other synthetic

likelihood methods, which either rely on normal-

izing ﬂows, or minimize score-based objectives;

choices that come with known pitfalls. We demon-

strate the properties of both methods on a range

of synthetic datasets, and apply them to a neuro-

science model of the pyloric network in the crab,

where our method outperforms prior art for a frac-

tion of the simulation budget.

1. Introduction

Simulation-based modeling expresses a system as a prob-

abilistic program (Ghahramani,2015), which describes,

in a mechanistic manner, how samples from the system

are drawn given the parameters of the said system. This

probabilistic program can be concretely implemented in a

computer - as a simulator - from which synthetic parameter-

samples pairs can be drawn. This setting is common in

many scientiﬁc and engineering disciplines such as stellar

events in cosmology (Alsing et al.,2018;Schafer & Free-

man,2012), particle collisions in a particle accelerator for

high energy physics (Eberl,2003;Sj

ostrand et al.,2008),

and biological neural networks in neuroscience (Markram

Gatsby Computational Neuroscience Unit, University Col-

lege London, London, UK

Universit

e Grenoble Alpes, CNRS,

Grenoble INP, LJK, 38000 Grenoble, France

Deepmind. Corre-

spondence to: Pierre Glaser <pierreglaser@gmail.com>.

et al.,2015;Pospischil et al.,2008). Describing such sys-

tems using a probabilistic program often turns out to be

easier than specifying the underlying probabilistic model

with a tractable probability distribution. We consider the

task of inference for such systems, which consists in com-

puting the posterior distribution of the parameters given

observed (non-synthetic) data. When a likelihood function

of the simulator is available alongside with a prior belief

on the parameters, inferring the posterior distribution of the

parameters given data is possible using Bayes’ rule. Tra-

ditional inference methods such as variational techniques

(Wainwright & Jordan,2008) or Markov Chain Monte Carlo

(Andrieu et al.,2003) can then be used to produce approxi-

mate posterior samples of the parameters that are likely to

have generated the observed data. Unfortunately, the likeli-

hood function of a simulator is computationally intractable

in general, thus making the direct application of traditional

inference techniques unusable for simulation-based mod-

elling.

Simulation-Based Inference (SBI) methods (Cranmer et al.,

2020) are methods speciﬁcally designed to perform infer-

ence in the setting of a simulator with an intractable like-

lihood. These methods repeatedly generate synthetic data

using the simulator to build an estimate of the posterior,

that either can be used for any observed data (resulting in

a so-called amortized inference procedure) or one that is

targeted for a speciﬁc observation. While the accuracy of

inference increases as more simulations are run, so does

computational cost, especially when the simulator is ex-

pensive, which is common in many physics applications

(Cranmer et al.,2020). In high-dimensional settings, early

simulation-based inference techniques such as Approximate

Bayesian Computation (ABC) (Marin et al.,2012) struggle

to generate high quality posterior samples at a reasonable

cost, since ABC repeatedly rejects simulations that fail to

reproduce the observed data (Beaumont et al.,2002). More

recently, model-based inference methods (Wood,2010;Pa-

pamakarios et al.,2019;Hermans et al.,2020;Greenberg

et al.,2019), which encode information about the simulator

via a parametric density (-ratio) estimator of the data, have

been shown to drastically reduce the number of simulations

needed to reach a given inference precision (Lueckmann

et al.,2021). The computational gains are particularly im-

arXiv:2210.14756v2 [cs.LG] 18 Apr 2023

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

portant when comparing ABC to targeted SBI methods, im-

plemented in a multi-round procedure that reﬁnes the model

around the observed data, by sequentially simulating data

points that are closer to the observed ones (Greenberg et al.,

2019;Papamakarios et al.,2019;Hermans et al.,2020).

Previous model-based SBI methods have used their para-

metric estimator to learn the likelihood (e.g. the condi-

tional density specifying the probability of an observation

being simulated given a speciﬁc parameter set, Wood 2010;

Papamakarios et al. 2019;Pacchiardi & Dutta 2022), the

likelihood-to-marginal ratio (Hermans et al.,2020), or the

posterior function directly (Greenberg et al.,2019). We fo-

cus in this paper on likelihood-based (also called Synthetic

Likelihood; SL, in short) methods, of which two main in-

stances exist: (Sequential) Neural Likelihood Estimation

(or (S)NLE) (Papamakarios et al.,2019), which learns a

likelihood estimate using a normalizing ﬂow trained by

optimizing a Maximum Likelihood (ML) loss; and Score

Matched Neural Likelihood Estimation (SMNLE Pacchiardi

& Dutta 2022), which learns an unnormalized (or Energy-

Based,LeCun et al. 2006) likelihood model trained using

conditional score matching. Recently, SNLE was applied

successfully to challenging neural data (Deistler et al.,2021).

However, limitations still remain in the approaches taken

by both (S)NLE and SMNLE. On the one hand, ﬂow-based

models may need to use very complex architectures to prop-

erly approximate distributions with rich structure such as

multi-modality (Kong & Chaudhuri,2020;Cornish et al.,

2020). On the other hand, score matching, the objective of

SMNLE, minimizes the Fisher Divergence between the data

and the model, a divergence that fails to capture important

features of probability distributions such as mode propor-

tions (Wenliang & Kanagawa,2020;Zhang et al.,2022).

This is unlike Maximimum-Likelihood based-objectives,

whose maximizers satisfy attractive theoretical properties

(Bickel & Doksum,2015).

Contributions.

In this work, we introduce Amortized Un-

normalized Likelihood Neural Estimation (AUNLE), and Se-

quential UNLE, a pair of SBI Synthetic Likelihood methods

performing respectively sequential and targeted inference.

Both methods learn a Conditional Energy Based Model of

the simulator’s likelihood using a Maximum Likelihood

(ML) objective, and perform MCMC on the posterior esti-

mate obtained after invoking Bayes’ Rule. While posteriors

arising from conditional EBMs exhibit a particular form of

intractability called double intractability, which requires the

use of tailored MCMC techniques for inference, we train

AUNLE using a new approach which we call tilting. This

approach automatically removes this intractability in the

ﬁnal posterior estimate, making AUNLE compatible with

standard MCMC methods, and signiﬁcantly reducing the

computational burden of inference. Our second method,

SUNLE, departs from AUNLE by using a new training

technique for conditional EBMs which is suited when the

proposal distribution is not analytically available. While

SUNLE returns a doubly intractable posterior, we show that

inference can be carried out accurately through robust im-

plementations of doubly-intractable MCMC or variational

methods. We demonstrate the properties of AUNLE and

SUNLE on an array of synthetic benchmark models (Lueck-

mann et al.,2021), and apply SUNLE to a neuroscience

model of the crab Cancer borealis, where we improve the

posterior accuracy over prior state-of-the-art while need-

ing only a fraction of the simulations required by the most

efﬁcient previous method (Gl¨

ockler et al.,2021).

Figure 1.

Performance of SMNLE, NLE and AUNLE, all trained

using a simulator with a bimodal likelihood

p(x|θ)

and a Gaussian

prior

p(θ)

, using 1000 samples. Top: simulator likelihood

p(x|θ0)

for some ﬁxed θ0. Bottom: posterior estimate.

2. Background

Simulation Based Inference (SBI) refers to the set of meth-

ods aimed at estimating the posterior

p(θ|xo)

of some un-

observed parameters

θ∈Θ⊂RdΘ

given some observed

variable

xo∈ X ⊂ RdX

recorded from a physical system,

and a prior

p(θ)

. In SBI, one assumes access to a simula-

tor

G: (θ, u)7−→ y=G(θ, u)

, from which samples

y|θ

can be drawn, and whose associated likelihood

p(y|θ)

accu-

rately matches the likelihood

p(x|θ)

of the physical system

of interest. Here,

represents draws of all random variables

involved in performing draws of

x|θ

. By a slight abuse of

notation, we will not distinguish between the physical ran-

dom variable

representing data from the physical system

of interest, and the simulated random variable

drawn from

the simulator: we will use

for both. The complexity of the

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

simulator (Cranmer et al.,2020) prevents access to a simple

form for the likelihood

p(x|θ)

, making standard Bayesian

inference impossible. Instead, SBI methods perform infer-

ence by drawing parameters from a proposal distribution

π(θ)

, and use these parameters as inputs to the simulator

to obtain a set of simulated pairs

(x, θ)

which they use

to compute a posterior estimate of

p(θ|x)

. Speciﬁc SBI

submethods have been designed to handle separately the

case of amortized inference, where the practitioner seeks to

obtain a posterior estimate valid for any

(which might

not be known a priori), and targeted inference, where the

posterior estimate should maximize accuracy for a speciﬁc

observed variable

. Amortized inference methods often

simply set their proposal distribution

to be the prior

whereas targeted inference methods iteratively reﬁne their

proposal

to focus their simulated observations around

the targeted

through a sequence of simulation-training

rounds (Papamakarios et al.,2019).

2.1. (Conditional) Energy-Based Models

Energy-Based Models (LeCun et al.,2006) are unnormal-

ized probabilistic models of the form

qψ(x) = e−Eψ(x)

Z(ψ), Z(ψ) = Ze−Eψ(x)dx,

where

Z(ψ)

is the intractable normalizing constant of the

model, and

Eψ

is called the energy function, usually set

to be a neural network with weights

. By directly mod-

elling the density

p(x)

of the data through a ﬂexible energy

function, simple EBMs can capture rich geometries and

multi-modality, whereas other model classes such a nor-

malizing ﬂows may require a more complex architecture

(Cornish et al.,2020). The ﬂexibility of EBMs comes at

the cost of having an intractable density

qψ(x)

due to the

presence of the normalizer

Z(ψ)

, increasing the challenge

of both training and sampling. In particular, an EBM’s log-

likelihood

log qψ

and associated gradient

∇ψlog qψ

both

contain terms involving the (intractable) normalizer

Z(ψ)

log qψ(x) = −Eψ(x)−

intractable

z }| {

log Z(ψ),

∇ψlog qψ(x) = −∇ψEψ(x) + Ex∼qψ∇ψEψ(x)

| {z }

intractable

,(1)

making exact training of EBMs via Maximum Likelihood

impossible. Approximate likelihood optimization can be

performed using a Gradient-Based algorithm where at each

iteration

, the intractable expectation (under the EBM

qψk

)

present in

∇ψlog qψk

is replaced by a particle approxima-

tion

bq=1

NPN

i=1 wiδyi

qψ

. The particles

y(i)

form-

ing

are traditionally set to be samples from a MCMC

chain with invariant distribution

qψk

, with uniform weights

wi=1

; recent work on EBM for high-dimensional image

data uses an adaptation of Langevin Dynamics (Raginsky

et al.,2017;Du & Mordatch,2019;Nijkamp et al.,2019;

Kelly & Grathwohl,2021). We outline the traditional ML

learning procedure for EBMs in Algorithm 4(Appendix),

where

make particle approx(q, ˆq0)

is a generic rou-

tine producing a particle approximation of a target unnor-

malized density qand an initial particle approximation ˆq0.

Energy-Based Models are naturally extended to both joint

EBMs

qψ(θ, x) = e−Eψ(θ,x)

Z(ψ)

(Kelly & Grathwohl,2021;

Grathwohl et al.,2020) and conditional EBMs (CEBMs

Khemakhem et al. 2020;Pacchiardi & Dutta 2022) of the

form:

qψ(x|θ) = e−Eψ(x,θ)

Z(θ, ψ), Z(θ;ψ) = Ze−Eψ(x,θ)dx. (2)

Unlike joint and standard EBMs, conditional EBMs deﬁne

a family of conditional densities

qψ(x|θ)

, each of which is

endowed with an intractable normalizer Z(θ, ψ).

2.2. Synthetic Likelihood Methods for SBI

Synthetic Likelihood (SL) methods (Wood,2010;Papa-

makarios et al.,2019;Pacchiardi & Dutta,2022) form a

class of SBI methods that learn a conditional density model

qψ(x|θ)

of the unknown likelihood

p(x|θ)

for every pos-

sible pair of observations and parameters

(x, θ)

. The set

{qψ(x|θ), ψ ∈Ψ}

is a model class parameterised by some

vector

ψ∈Ψ

, which recent methods set to be a neural net-

work with weights

. We describe the existing Neural SL

variants to date.

Neural Likelihood Estimation

(NLE, Papamakarios et al.

2019) sets

qψ

to a (normalized) ﬂow-based model, and

is optimized by maximizing the average conditional log-

likelihood

Eπ(θ)p(x|θ)log qψ(x|θ)

. NLE performs inference

by invoking Bayes’ rule to obtain an unnormalized pos-

terior estimate

pψ(θ|x) = qψ(x|θ)p(θ)

Rqψ(x|θ)p(θ)dθ∝p(θ)qψ(x|θ)

from which samples can be drawn either using MCMC, or

Variational Inference (Gl¨

ockler et al.,2021).

Score Matched Neural Likelihood Estimation

(SMNLE,

Pacchiardi & Dutta 2022) models the unknown likelihood

using a conditional Energy-Based Model

qψ(x|θ)

of the

form of Equation (2), trained using a score matching objec-

tive adapted for conditional density estimation. The use of

an unnormalized likelihood model makes the posterior esti-

mate obtained via Bayes’ Rule known up to a

-dependent

term:

qψ(θ|x)∝p(θ)qψ(x|θ)∝e−Eψ(x,θ)p(θ)

Z(θ, ψ)

| {z }

intractable

Z(θ, ψ) = Ze−Eψ(x,θ)dx.

(3)

Posteriors of this form are called doubly intractable pos-

teriors (Møller et al.,2006), and can be sampled from a

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

subclass of MCMC algorithms designed speciﬁcally to han-

dle doubly intractable distributions (Murray et al.,2006;

Møller et al.,2006).

Both the likelihood objective of NLE and the score-based

objective of SMNLE do not involve the analytic expression

of the proposal

, making it easy to adapt these methods

for either amortized or targeted inference. To address the

limitations of both methods mentioned in the introduction,

we next propose a method that combines the use of ﬂexible

Energy-Based Models as in SMNLE, while being optimized

using a likelihood loss as in NLE.

3. Unnormalized Neural Likelihood

Estimation

In this section, we present our two methods, Amortized-

UNLE and Sequential-UNLE. Both AUNLE and SUNLE

approximate the unknown likelihood

p(x|θ)

for any possi-

ble pair of

(x, θ)

using a conditional Energy-Based Model

qψ(x|θ)

as in Equation (2), where

Eψ

is some neural net-

work. Additionally, AUNLE and SUNLE are both trained

using a likelihood-based loss; however, the training objec-

tives and inference phases differ to account for the speciﬁci-

ties of amortized and targeted inference, as detailed below.

3.1. Amortized UNLE

Given a likelihood model

qψ(x|θ)

, a natural learning proce-

dure would involve ﬁtting a model

qψ(x|θ)π(θ)

of the true

“joint synthetic” distribution

π(θ)p(x|θ)

, as NLE does. We

show, however, that using an alternative – tilted – version

of this model allows to compute a posterior that is more

tractable than those computed by other SL methods relying

on conditional EBMs such as SMNLE (Pacchiardi & Dutta,

2022). Our method, AUNLE, ﬁts a joint probabilistic model

qψ,π of the form:

qψ,π(x, θ) := π(θ)e−Eψ(x,θ)

Zπ(ψ),

Zπ(ψ) = Zπ(θ)e−Eψ(x,θ)dxdθ.

(4)

by maximizing its log-likelihood

La(ψ) :=

Eπ(θ)p(x|θ)[log qψ,π(x, θ)]

using an instance of Algo-

rithm 4. The gain in tractability offered by AUNLE is a

direct consequence of the following proposition.

Proposition 3.1.

Let

Pψ:= {qψ(·|θ), ψ ∈Ψ}

, and

qψ∈

Pψ. Then we have:

•(likelihood modelling) qψ,π (x|θ) = qψ(x|θ)

•

(joint model tilting)

qψ,π(x, θ) = f(θ)π(θ)qψ(x|θ)

for f(θ) := Z(θ, ψ)/Zπ(ψ),and Z(θ, ψ)from (2)

•

((Z,

)-uniformization) If

p(·|θ)∈ Pψ

, then the

ψ?

minimizing

La(ψ)

satisﬁes:

qψ?(x|θ) = p(x|θ)

, and

Z(θ, ψ?) = Zπ(ψ?).

Proof.

The ﬁrst point follows by holding

ﬁxed in

qψ,π(x, θ)

. To prove the second point, notice that

qψ,π(x, θ) = Z(θ,ψ)

Z(θ,ψ)

π(θ)e−E(x,θ)

Zπ(ψ)=Z(θ,ψ)

Zπ(ψ)π(θ)e−E(x,θ)

Z(θ,ψ)

For the last point, note that at the optimum, we have that

qψ?,π(x, θ) = π(θ)p(x|θ)

. Integrating out

on both sides

of the equality yields

f(θ)π(θ) = π(θ)

, proving the re-

sult.

Proposition 3.1 shows that AUNLE indeed learns a like-

lihood model

qψ(x|θ)

through a joint model

qψ,π

tilting

the prior

with

f(θ)

. Importantly, this tilting guarantees

that the optimal likelihood model will have a normaliz-

ing function

Z(θ;ψ)

constant (or uniform) in

, reducing

AUNLE’s posterior to a standard unnormalized posterior

qψ?(θ|x) = p(θ)e−Eψ?(θ,x)

Zπ(ψ?)

. AUNLE then performs infer-

ence using classical MCMC algorithms targetiting

qψ

. The

standard nature of AUNLE’s posterior contrasts with the

posterior of SMNLE (Pacchiardi & Dutta,2022), and al-

lows to expand the range of inference methods applicable

to it, which otherwise would have been restricted to MCMC

methods for doubly intractable distributions. In particular,

the sampling cost of inference could be further reduced by

performing a Variational Inference step such as in (Gl

ockler

et al.,2021). Whether or not the

(Z, θ)

-uniformity holds

will depend on the degree to which

qψ,π(x|θ)

correctly mod-

els

p(x|θ)

. This is particularly difﬁcult when

p(x|θ)

is a

“complicated” function of

(e.g. non-smooth, diverging).

We further investigate this scenario when it arises in our

experiments (see the SLCP model and Appendix C.3).

Algorithm 1 Amortized-UNLE

Input:

prior

p(θ)

, simulator

, budget

, initial EBM

parameters ψ0

Output: posterior estimate qψ(θ|x)

Initialize π=p, qψ0,π ∝e−Eψ0(x,θ)π(θ)

for i= 0, . . . , N do

Draw θ∼π,x∼G(θ, ·)

Add (θ, x)to D

end for

Get ψ?:= maximize ebm log l(D, ψ0)

Set qψ?(θ|x) := e−Eψ?(x,θ)p(θ)

Infer using MCMC on qψ?(θ|x)

3.2. Targeted Inference using Sequential-UNLE

In this section, we introduce our second method, Sequential-

UNLE (or SUNLE in short), which performs targeted infer-

ence for a speciﬁc observation

. SUNLE follows the tra-

ditional methodology of targeted inference by splitting the

simulator budget

over

rounds (often equally), where in

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

each round

, a likelihood estimate

qψ?

r(x|θ)

in the form of a

conditional EBM is trained using all the currently available

simulated data

. This allows to construct a new poste-

rior estimate

qψ?

r(θ|x)=e−Eψ?

r(x,θ)p(θ)/Z(θ, ψ?

which is

used to sample parameters

{θ(i)}N/R

i=1

that are then provided

to the simulator for generating new data

xi∼G(θ(i))

. The

new data are added to the set

and are expected to be more

similar to the observation of interest

. This procedure

allows to focus the simulator budget on regions relevant

to the single observed data of interest

, and, as such, is

expected to be more efﬁcient in terms of the simulator use

than amortized methods (Lueckmann et al.,2021). Next, we

discuss the learning procedure for the likelihood model and

the posterior sampling.

3.2.1. LEARNING THE LIKELIHOOD

At each round

, the effective proposal

of the

training data available can be understood (provided

the number of data points drawn at reach rounds

is randomized) as a mixture probability:

π:=

r(π(0)(θ)+qψ?

1(θ|xo)+ . . . +qψ?

r−1(θ|xo))

which is used

to update the likelihood model. In this case, the analytical

form of

is unavailable as it requires computing the nor-

malizing constants of the posterior estimates at each round,

thus making the tilting approach introduced for AUNLE

impractical in the sequential setting. Instead, SUNLE learns

a likelihood model maximizing the average conditional log-

likelihood,

L(ψ) = 1

i=1

log qψ(xi|θi),(5)

where

(xi, θi)N

i=1

are the current samples. Unlike standard

EBM objectives, this loss directly targets the likelihood

qψ(x|θ)

, thus bypassing the need for modelling the proposal

. We propose

maximize cebm log l

(Algorithm 7,

Appendix), a method that optimizes this objective (previ-

ously used for normalizing ﬂows in Papamakarios et al.,

2019) when the density estimator is a conditional EBM. The

intractable term of Equation (5) is an average over the EBM

probabilities conditioned on all parameters from the training

set, and thus differs from the intractable term of

(1)

, com-

posed of a single integral. Algorithm 7approximates this

term during training by keeping track of one particle approx-

imation

bqi=δ˜xi

per conditional density

qψ(·|θi)

comprised

of a single particle. The algorithm proceeds by updating

only a batch of size

of such particles using an MCMC

update with target probability chain

qψk(·|θi)

, where

ψk

is the EBM iterate at iteration

of round

. Learning the

likelihood using Algorithm 7allows to use all the exist-

ing simulated data during training without re-learning the

proposal, maximizing sample efﬁciency while minimizing

learning complexity. The multi-round procedure of SUNLE

is summarized in Algorithm 2.

Algorithm 2 Sequential-UNLE

Input: prior p(θ), simulator G, budget N, no. rounds R

Output: Posterior estimate qψ(θ|x)

Initialize π(0) =p, ψ∗

0=ψ0, qψ0,π ∝e−Eψ0(x,θ)π(θ), w0

Get {θ(i)∼π(θ)}N/R

i=1 , set D={θ(i), x(i)∼G(θ, ·)}N/R

i=1

for r= 1, . . . , R do

Get qψ?

r(x|θ):=maximize cebm log l(D, qψ?

r−1)

Set qψ?

r(θ|x)∝p(θ)qψ?(x|θ)

Get

qψ∗

r(θ|x),{θ(i)}N/R

i=1

via Doubly-Intr. MCMC or

DIVI+MCMC (Explained in Section 3.2.2)

Set D=D ∪ {θ(i), x(i)∼G(θ(i),·)}N/R

i=1

end for

Return qψ?

R(θ|x)

3.2.2. POSTERIOR SAMPLING

Unlike AUNLE, SUNLE’s likelihood estimate

qψ?

R(·|θ)

does not inherit the

(Z, θ)

-uniformization property guar-

anteed by Proposition 3.1. As a consequence, its posterior

qψ?

R(θ|x)

is doubly intractable as it contains an intractable

-dependent term

Z(ψ?

R, θ)

. We discuss two methods to

sample from

qψ?

R(θ|x)

: Doubly Intractable MCMC, and a

two-step approach which performs MCMC on a “singly in-

tractable” approximation of the doubly intractable posterior.

Doubly Intractable MCMC

methods (Møller et al.,2006;Murray et al.,2006) are

MCMC algorithms that can generate samples from a dou-

bly intractable posterior. They consist in running a stan-

dard MCMC algorithm targeting an augmented distribu-

tion

p(θ, yaux|x)

whose marginal in

equals the posterior

qψ(θ|x)

: approximate posterior samples are obtained by se-

lecting the

component of the augmented samples returned

by the MCMC algorithm while throwing away the auxiliary

part. Importantly, such MCMC algorithms need to sample

from the likelihood

qψ(x|θ)

at every iteration to compute the

acceptance probability of the proposed augmented sample.

As SUNLE’s likelihood

qψ(x|θ)

cannot be tractably sam-

pled exactly, our implementation proceeds as in (Pacchiardi

& Dutta,2022;Everitt,2012;Alquier et al.,2016) and re-

places exact likelihood sampling by approximate sampling

using MCMC.

Doubly Intractable Variational Inference

While sam-

ples returned by doubly intractable MCMC algorithms often

accurately estimate their target (Pacchiardi & Dutta,2022;

Everitt,2012;Alquier et al.,2016), working with doubly

intractable posteriors nonetheless complicates the task of

inference: the increased computational cost arising from run-

ning an inner MCMC chain targeting

qψ(x|θ)

limits the total

number of posterior samples obtainable given a reasonable

time budget. Additionally the shape of pairwise conditionals

(Gl¨

ockler et al.,2021)p(θi, θj|θk6=i,j , x), available when p

is a standard unnormalized posterior, becomes inaccessible

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MaximumLikelihoodLearningofUnnormalizedModelsforSimulation-BasedInferencePierreGlaser1MichaelArbel2SamoHromadka1ArnaudDoucet3ArthurGretton1AbstractWeintroducetwosyntheticlikelihoodmethodsforSimulation-BasedInference(SBI),toconducteitheramortizedortargetedinferencefromexper-imentalobservationswhenahi...

展开>> 收起<<

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference.pdf

共23页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: