
Towards Explaining Distribution Shifts
k
-sparse transport is most useful in situations where a dis-
tribution shift has happened along a subset of dimensions,
such as explaining a shift where some sensors in a network
are picking up a change in an environment. However, in
cases where points shift in different directions based on their
original value, e.g. when investigating how a heterogeneous
population responded to an advertising campaign,
k
-sparse
transport is not ideal. Thus, we provide a shift explanation
that breaks the source and target distributions into
k
sub-
populations and provides a vector-based shift explanation
per sub-population, which we call k-Cluster Transport.
k
-Cluster Transport: Given a
k∈ {1, . . . , D}
we
define
k
-cluster transport to be a mapping which moves
each point
x
by constant vector which is specific to
x
’s
cluster. More formally, we define a labeling function
σ(x;M)≜arg min j∥mj−x∥2
, which returns the
index of the column in
M
(i.e. the label of the cluster)
which
x
is closest to. With this, we define
Ω(k)
cluster =
T:T(x) = x+δσ(x;M), M ∈RD×k,∆∈RD×k
,
where δjis the jth column of ∆.
Since measuring the exact interpretability of a mapping
is heavily context-dependent, we can instead use
k
in the
above transport maps to define a partial ordering of inter-
pretability of mappings within a class of transport maps. Let
k1
and
k2
be the size of the active sets for
k
-sparse maps
(or the number of clusters for
k
-cluster maps) of
T1
and
T2
respectively. If
k1≤k2
, then
Inter(T1)≥Inter(T2)
, where
Inter(T)
is the interpretability of shift explanation
T
. For
example, we claim the interpretability of a
T1∈Ω(k=10)
sparse
is
greater than (or possibly equal to) the interpretability of a
T2∈Ω(k=100)
sparse
since a shift explanation in
Ω
which moves
points along only 10 dimensions is more interpretable than a
similar mapping which moves points along 100 dimensions.
A similar result can be shown for
k
-cluster transport since
an explanation of how 5 clusters moved under a shift is less
complicated than an explanation of how 10 clusters moved.
The above method allows us to define a partial ordering on
interpretability without having to determine the absolute
value of interpretability of an individual explanation
T
, as
this requires expensive context-specific human evaluations,
which are out of scope for this paper.
3.3. Intrinsically Interpretable Maps For Images
To find interpretable transport mappings for images, we
could first project
Psrc
and
Ptgt
onto a low-dimensional in-
terpretable latent space (e.g., a space which has disentangled
and semantically meaningful dimensions) and then apply the
methods above in this latent space. Concretely, let us denote
the
(pseudo-)invertible
encoder as
g:RD→RD′
where
D′< D
(e.g., an autoencoder). Given this encoder, we de-
fine our set of high dimensional interpretable transport maps:
Ωhigh-dim :=nT:T=g−1˜
T(g(x)),˜
T∈Ω, g ∈ Io
where
Ω
the set of interpretable mappings (e.g.,
k
-sparse
mappings) and
I
is the set of
(pseudo-)invertible
functions
with an interpretable (i.e. semantically meaningful) latent
space. Finally, given an interpretable
g∈ I
, this gives us
High-dimensional Interpretable Transport:THIT .
As seen in the Stanford Wilds dataset (Koh et al.,2021),
which contains benchmark examples of real-world image-
based distribution shifts, image-based shifts can be im-
mensely complex. In order to provide an adequate intrin-
sically interpretable mapping explanation of a distribution
shift in high dimensional data (e.g., images), multiple new
advancements must first be met (e.g., finding a disentangled
latent space with semantically meaningful dimensions, ap-
proximating high dimensional empirical optimal transport
maps, etc.), which are out of scope of this paper. We further
explore details about
THIT
, its variants, and the results of
using
THIT
to explain Colorized-MNIST in Appendix D,
and we hope future work could build upon this framework.
3.4. Post-Hoc Explanations of Image-Based Mappings
via Counterfactual Examples
As mentioned above, in some cases, solving for an inter-
pretable latent space can be too difficult or costly, and thus
a shift cannot be expressed by an interpretable mapping
function. However, if the samples themselves are easy to
interpret (e.g., images), we can still explain a transport map-
ping by visualizing translated samples. Specifically, we can
remove the interpretable constraint on the mapping itself
and leverage methods from the unpaired Image-to-Image
translation (I2I) literature to translate between the source
and target domain while preserving the content. For a com-
prehensive summary of the recent I2I works and methods,
please see (Pang et al.,2021).
Once an I2I mapping is found, to serve as an explanation,
we can provide an operator with a set of counterfactual pairs
{(x, T (x)) : x∼Psrc, T (x)∼Ptgt}
. Then, by determin-
ing what commonly stays invariant and what commonly
changes across the set of counterfactual pairs, this can serve
as an explanation of how the source distribution shifted to
the target distribution. While more broadly applicable, this
approach could put a higher load on the operator than an
intrinsically interpretable mapping approach.
4. Practical Methods for Finding and
Validating Shift Explanations
In this section, we discuss practical methods for finding
these maps via empirical OT (Sec. 4.1,4.2, and 4.3) and
introduce a PercentExplained metric which can assist the
operator in selecting the hyperparameter
k
in
k
-sparse and
k-cluster transport (Sec. 4.4).
4