Approved for public release distribution unlimited. Star-Graph Multimodal Matching Component Analysis for Data Fusion and Transfer Learning

2025-04-27 0 0 422.09KB 17 页 10玖币

侵权投诉

Approved for public release: distribution unlimited.

Star-Graph Multimodal Matching Component Analysis

for Data Fusion and Transfer Learning

Nick Lorenzo

August 5, 2022

Abstract

Previous matching component analysis (MCA) techniques map two data domains to a common domain

for further processing in data fusion and transfer learning contexts. In this paper, we extend these

techniques to the star-graph multimodal (SGM) case in which one particular data domain is connected

to mothers via an objective function. We provide a particular feasible point for the resulting trace

maximization problem in closed form and algorithms for its computation and iterative improvement,

leading to our main result, the SGM maps. We also provide numerical examples demonstrating that

SGM is capable of encoding into its maps more information than MCA when few training points are

available. In addition, we develop a further generalization of the MCA covariance constraint, eliminating

a previous feasibility condition and allowing larger values of the rank of the prescribed covariance matrix.

1 Introduction

The matching component analysis (MCA) technique for transfer learning [1] ﬁnds two maps – one from each

of two data domains to a lower-dimensional, common domain – using only a small number of matched data

pairs, where each matched data pair is comprised of one point from each data domain. These maps minimize

the expected distance between mapped data pairs within the common domain, subject to an identity matrix

covariance constraint and an aﬃne linear structure. Learning techniques can then be applied to matched

data points after they are mapped to the common domain, where each such point is encoded with information

from both data domains via its respective optimal aﬃne linear transformation.

In [2], the covariance-generalized MCA (CGMCA) technique was developed in order to allow for the encoding

of additional statistical information into the MCA maps. This was done by generalizing the identity matrix

covariance constraint of MCA to accommodate any covariance matrix (compare Figures 1a and 1b).

We are interested in extending the application space of CGMCA to accommodate three or more data domains

simultaneously. In this paper, we restrict our attention to the star-graph case of this multimodal1general-

ization: one central modality is mapped into the common domain as closely as possible to each of the other

modalities, with no two non-central modalities mapped together in this way. These close-mapping conditions

appear as norms in the objective function (5a), and this norm-connectedness is represented schematically by

dotted lines in Figure 1. We refer to this star-graph multimodal generalization of CGMCA as SGM.

Multimodal data can already be handled one way by MCA and CGMCA: matched, vectorized data points

from two or more modalities can be stacked on top of one another, forming higher-dimensional data points [3].

These higher-dimensional data points can then be treated as though they come from a single modality, with

MCA or CGMCA applied as usual; this is illustrated in Figure 4c, where the ﬁrst modality is unstacked and

University of Dayton Research Institute, Dayton, OH, USA (Nicholas.Lorenzo@udri.udayton.edu)

1We use the terms “data domain” and “modality” interchangeably.

Approved for public release: distribution unlimited.

two sub-modalities are stacked into the second modality. This approach, however, provides no immediately

apparent mechanism for weighting the inﬂuence of each sub-modality in the resulting maps. It also eﬀectively

forces each set of stacked sub-modalities to directly compete with one another for representation in the

common domain, as all stacked sub-modalities are represented by a single MCA map. In addition, this

approach enforces a single covariance structure onto the common-domain image of the stacked sub-modalities,

rather than allowing each sub-modality to have its own prescribed covariance. In order to overcome these

limitations, we develop SGM.

2 Mathematical development of SGM

Our development of the SGM optimization problem mirrors the development of the MCA [1] and CGMCA

[2] optimization problems, with some notation borrowed from [4, p. 130].

2.1 Notation

For x∈N, let [x]≡ {1, . . . , x}and let [x]0≡ {0, . . . , x}= [x]∪ {0}.

Let a, b, c ∈Nwith c≤min{a, b},v∈Rc, and A∈Ra×b. Deﬁne

diaga×b(v)∈Ra×b(1a)

to be the matrix whose ﬁrst cdiagonal elements are the elements of v(in order) and zero elsewhere, and

deﬁne

diag−1

c(A)∈Rc(1b)

to be the vector comprised of the ﬁrst cdiagonal elements of the matrix A(in order). We also deﬁne

dgc(A)≡diagc×cdiag−1

c(A)∈Rc×c(1c)

to be the diagonal matrix whose diagonal entries are the ﬁrst cdiagonal elements of the matrix A.

Let

Oa×b≡ {O∈Ra×b|OTO=Ib}(2a)

denote the set of a×breal matrices with orthonormal columns (this set is non-empty only if b≤a; when

b=a, these matrices are orthogonal), let

+≡ {D∈Ra×a|D= diaga×a[δ1, . . . , δa]T∃δ1≥. . . ≥δa>0}(2b)

denote the set of diagonal a×areal invertible matrices with non-increasing elements, and let

Da×b

̸− ≡ {D∈Ra×b|D= diaga×b[δ1, . . . , δmin{a,b}]T∃δ1≥. . . ≥δmin{a,b}≥0}(2c)

denote the set of diagonal a×breal matrices with non-negative, non-increasing elements.

We reserve the undecorated but subscripted symbols U(·), Σ(·), and V(·)and barred, subscripted symbols

U(·), Σ(·), and V(·)exclusively for use as factors in the SVD. In particular, for a matrix Z∈Rrows(Z)×cols(Z),

we denote an arbitrary but ﬁxed thin SVD of Zby

Z=UZΣZVT

Z,where UZ∈Orows(Z)×rank(Z),ΣZ∈Drank(Z)

+, VZ∈Ocols(Z)×rank(Z),(3a)

and we denote a ﬁxed full SVD of Zby

Z=UZΣZVT

Z,where UZ∈Orows(Z)×rows(Z),ΣZ∈Drows(Z)×cols(Z)

̸− , VZ∈Ocols(Z)×cols(Z),(3b)

Approved for public release: distribution unlimited.

common

domain

data

domain 0

data

domain 1

g0g1

∥g0−g1∥

(a) MCA

common

domain

data

domain 0

data

domain 1

g0g1

∥g0−g1∥

(b) CGMCA

common

domain

data

domain 0

data

domain 1

data

domain 2

data

domain 3

data

domain m

λ1∥g0−g1∥

λ2∥g0−g2∥

λ3∥g0−g3∥

λm∥g0−gm∥

Figure 1: Schematic representation of the norm-connectedness and mappings of (a) MCA, (b) CGMCA,

and (c) SGM. Each data domain maps into the common domain (arrows); data domain 0 is norm-connected

to each of the other data domains (dotted lines). The dashed arrows for MCA indicate its restriction to

an identity covariance constraint; the solid arrows for CGMCA indicate its ability to prescribe covariance

information; the thick arrows for SGM represent its further generalized covariance constraint (see Section

2.4). In SGM, the dotted lines form a star graph with data domain 0 at its center.

with UZan arbitrary but ﬁxed orthogonal completion of UZ, with VZan arbitrary but ﬁxed orthogonal

completion of VZ, and with ΣZhaving ΣZas a leading principal submatrix with zeros elsewhere.

We also reserve the subscripted and superscripted symbol B(·)

(·)to denote by Bℓ

Zthe best approximation (in

Approved for public release: distribution unlimited.

the Frobenius norm) of Zhaving rank at most ℓ[5, p. 36], where ℓ∈[rank(Z)]:

Bℓ

Z≡X

j∈[ℓ]

(ΣZ)jj (UZ)j(VZ)T

j∈Rrows(Z)×cols(Z),(4a)

with ( ·)jdenoting the jth column of a matrix and where

UBℓ

Z= (UZ)[ℓ]∈Orows(Z)×ℓ,ΣBℓ

Z= dgℓ(ΣZ)∈Dℓ

+, VBℓ

Z= (VZ)[ℓ]∈Ocols(Z)×ℓ,(4b)

with ( ·)[ℓ]denoting the ﬁrst ℓcolumns of a matrix.

2.2 Problem statement

For some m∈N, we consider random variables {Xi}i∈[m]0, each with domain probability space (Ω,M, P ),

and a set of outcomes {ωj}j∈[n]⊆Ω for some n∈Nwith 1 < n, and we are provided, for each i∈[m]0,

with the data set {Xi(ωj)}j∈[n]of independent random variates belonging to the ith data domain ΩXi=Rdi

for some di∈Nwith 1 < di. For each j∈[n], we refer to the length-(m+ 1) tuple of realizations

(X0(ωj), . . . , Xm(ωj)) as a matched data point. Provided a covariance matrix CiCT

i∈Rk×kfor each i∈[m]0

and for some ﬁxed k∈N, we wish to use the matched data to approximately solve the weighted, constrained

optimization problem

minimize E

X

i∈[m]

λi∥g0(X0)−gi(Xi)∥2

2

,(5a)

subject to gi:Rdi→Rk,(5b)

Egi(Xi)=0,(5c)

Ehgi(Xi)gi(Xi)Ti=CiCT

i,(5d)

i∈[m]0,

where the weights {λi}i∈[m]are ﬁxed, non-negative, and sum to unity. The fact that the map g0is paired

with each of the other maps {gi}i∈[m]in the objective function (5a) is reﬂected by the dotted lines of Figure

1c.

The optimization problem (5) is our multimodal generalization of the optimization problem (1) of [2] (in which

m≡1, represented schematically in Figure 1b), itself a covariance-generalized version of the optimization

problem (2.1) of [1] (in which both m≡1 and CiCT

i≡Ik, represented schematically in Figure 1a); we

seek to minimize the sum of weighted expected distances between the images under all pairs {(g0, gi)}i∈[m]

of the random variables {(X0, Xi)}i∈[m]within the common domain of dimension k, while centering the

resulting mappings {gi}i∈[m]0via (5c) and preserving the prescribed covariance structures {CiCT

i}i∈[m]0of

(5d).

In this star-graph version of the problem, we have one modality (indexed as the 0th modality) at the center

of the star graph, with mother modalities each norm-connected to the center modality but not to each

other. Thus the map g0is directly inﬂuenced by each of the maps {gi}i∈[m], while the maps {gi}i∈[m]have

no direct inﬂuence over one another (see the dotted lines of Figure 1c for a schematic representation of this

norm-connectedness).

2.3 Approximation via available data

Restricting gito be an aﬃne linear transformation, the requirement (5b) is equivalent to the requirement

that gi(x) = Aix+bifor some Ai∈Rk×diand bi∈Rk. Denoting by ˆµiand ˆ

Θithe mean and covariance,

respectively, of Xi, the centering constraint (5c) then implies the condition bi=−Aiˆµi, so that

gi(x) = Ai(x−ˆµi).(6)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Approvedforpublicrelease:distributionunlimited.Star-GraphMultimodalMatchingComponentAnalysisforDataFusionandTransferLearningNickLorenzo*August5,2022AbstractPreviousmatchingcomponentanalysis(MCA)techniquesmaptwodatadomainstoacommondomainforfurtherprocessingindatafusionandtransferlearningcontexts.Inth...

展开>> 收起<<

Approved for public release distribution unlimited. Star-Graph Multimodal Matching Component Analysis for Data Fusion and Transfer Learning.pdf

共17页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Approved for public release distribution unlimited. Star-Graph Multimodal Matching Component Analysis for Data Fusion and Transfer Learning

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: