Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes

2025-05-06 0 0 2.07MB 10 页 10玖币

侵权投诉

Modeling Inter-Dependence Between Time and Mark

in Multivariate Temporal Point Processes

Govind Waghmare

Mastercard, AI Garage

Gurugram, India

govind.waghmare@mastercard.com

Ankur Debnath

Mastercard, AI Garage

Gurugram, India

ankur.debnath@mastercard.com

Siddhartha Asthana

Mastercard, AI Garage

Gurugram, India

siddhartha.asthana@mastercard.com

Aakarsh Malhotra

Mastercard, AI Garage

Gurugram, India

aakarsh.malhotra@mastercard.com

ABSTRACT

Temporal Point Processes (TPP) are probabilistic generative frame-

works. They model discrete event sequences localized in continu-

ous time. Generally, real-life events reveal descriptive information,

known as marks. Marked TPPs model time and marks of the event

together for practical relevance. Conditioned on past events, marked

TPPs aim to learn the joint distribution of the time and the mark

of the next event. For simplicity, conditionally independent TPP

models assume time and marks are independent given event his-

tory. They factorize the conditional joint distribution of time and

mark into the product of individual conditional distributions. This

structural limitation in the design of TPP models hurt the predictive

performance on entangled time and mark interactions. In this work,

we model the conditional inter-dependence of time and mark to

overcome the limitations of conditionally independent models. We

construct a multivariate TPP conditioning the time distribution

on the current event mark in addition to past events. Besides the

conventional intensity-based models for conditional joint distri-

bution, we also draw on exible intensity-free TPP models from

the literature. The proposed TPP models outperform conditionally

independent and dependent models in standard prediction tasks.

Our experimentation on various datasets with multiple evaluation

metrics highlights the merit of the proposed approach.

CCS CONCEPTS

•Information systems →Location based services.

KEYWORDS

multivariate temporal point processes; probabilistic modeling

ACM Reference Format:

Govind Waghmare, Ankur Debnath, Siddhartha Asthana, and Aakarsh

Malhotra. 2022. Modeling Inter-Dependence Between Time and Mark in

Multivariate Temporal Point Processes. In Proceedings of the 31st ACM

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than the

author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or

republish, to post on servers or to redistribute to lists, requires prior specic permission

and/or a fee. Request permissions from permissions@acm.org.

CIKM ’22, October 17–21, 2022, Atlanta, GA, USA

ACM ISBN 978-1-4503-9236-5/22/10. . . $15.00

https://doi.org/10.1145/3511808.3557399

International Conference on Information and Knowledge Management (CIKM

’22), October 17–21, 2022, Atlanta, GA, USA. ACM, New York, NY, USA,

10 pages. https://doi.org/10.1145/3511808.3557399

Independent time and mark

Time

distribution

RNN

Conditionally independent models Proposed models

Mark

distribution

Multivariate TPP where time is

conditioned (dependent) on each mark

RNN

Mark

distribution

Input event

sequence

Figure 1: The proposed models are conditionally dependent,

multivariate, and capable of employing both intensity-free

and intensity-based formulations.

1 INTRODUCTION

TPP is a random process representing irregular event sequences

occurring in continuous time. Financial transactions, earthquakes,

and electronic health records (EHR) exhibit asynchronous temporal

patterns. TPPs are well studied in the literature and have rich theo-

retical foundations [

]. Classical (non-neural) TPPs focus on

capturing relatively simple temporal patterns through Poison pro-

cess [

], self-excitation process [

], and self-correcting process

[

]. With the advent of neural networks, many exible and ecient

neural architectures have been developed to model multi-modal

event dynamics, called neural TPPs [30].

Any attribute associated with an event makes it more realistic

and represented as a mark. Marks capture a better description of the

event, like time and location, interacting entities, and their evolu-

tion. Stochastic modeling of such events to study underlying event

generation mechanisms is called the marked TPPs. For instance, in

seismology, earthquake event dynamics are better understood with

the knowledge of magnitude and location [

]. A temporal model

solely learned on time may not be of practical relevance where

arXiv:2210.15294v2 [cs.LG] 23 Nov 2024

CIKM ’22, October 17–21, 2022, Atlanta, GA, USA Govind Waghmare, Ankur Debnath, Siddhartha Asthana, & Aakarsh Malhotra

marks impart realistic and reliable information. Marked TPP is a

probabilistic framework [

] which aims to model the joint distri-

bution of time and mark of the next event using previous event

history. An estimation of the next event time and the mark has

practical application in many domains that exhibit complex time

and mark interactions. Such application include online user engage-

ments [

], information diusion [

], econometrics [

], and

healthcare [

]. In personalized healthcare, a patient could have

a complex medical history, and several diseases may depend on

each other. Predictive EHR modeling could reveal potential future

clinical events and facilitate ecient resource allocation.

Time and mark dependency: While modeling the conditional

joint distribution of time and marks, many prior works assume

marks to be conditionally independent of time [

]. This assump-

tion on the conditional joint distribution of time and mark leads

to two types of marked TPPs, (i) conditionally independent, and

(ii) conditionally dependent models. The independence assump-

tion allows factorization of the conditional joint distribution into

a product of two independent conditional distributions. It is the

product of continuous-time distribution and categorical mark distri-

bution

, both conditioned on the event history. The independence

between time and mark limits the structural design of the neural ar-

chitecture in conditionally independent models. Thus, such models

require fewer parameters to specify the conditional joint distribu-

tion of time and marks but fail to capture their dependence. On the

contrary, conditionally dependent models capture the dependency

between time and mark by either conditioning time distribution on

mark or mark distribution on time. A recent study by [

] shows

that the conditionally independent models perform poorly com-

pared to conditionally dependent models.

Multivariate TPP: Marked TPP is a joint probability distribution

over a given time interval. In order to model time and mark depen-

dency, the time distribution should be conditioned on all possible

marks. It leads to a multivariate TPP model where a tuple of time

distributions is learned over a set of categorical marks [

]. For

𝐾

distinct marks,

𝑘𝑡ℎ

multivariate distribution (

𝑘∈ {

, . . . , 𝐾}

)

indicates the joint distribution of the time and the 𝑘𝑡ℎ mark.

Intensity-based vs intensity-free modeling: In both condition-

ally independent and conditionally dependent models, inter-event

time distribution is a key factor of the joint distribution. The stan-

dard way of learning time distribution is by estimating conditional

intensity function. However, the intensity function requires select-

ing good parametric formulation [

]. The parametric intensity

function often makes assumptions about the latent dynamics of

the point process. A simple parametrization has limited expres-

siveness but makes likelihood computation easy. Though an ad-

vanced parametrization adequately captures event dynamics, likeli-

hood computation often involves numerical approximation using

Newton-Raphson or Monte Carlo (MC). Besides intensity-based

formulation, other ways to model conditional inter-event time dis-

tribution involve probability density function (PDF) modeling, cu-

mulative distribution function, survival function, and cumulative

intensity function [

]. A model based on an intensity-free

focuses on closed-form likelihood, closed-form sampling, and exi-

bility to approximate any distribution.

1Categorical marks are conventional in the prior works.

In this work, we model inter-dependence between time and mark

by learning conditionally dependent distribution. While inferring

the next event, we model a PDF of inter-event time distribution for

each discrete mark. The time distribution conditioned on marks

improves the predictive performance of the proposed models com-

pared to others. A high-level overview of our approach is shown in

Figure 1. In summary, we make the following contributions:

•

We overcome the structural design limitation of condition-

ally independent models by proposing novel conditionally

dependent, both intensity-free and intensity-based, and multi-

variate TPP models. To capture inter-dependence between

mark and time, we condition the time distribution on the

current mark in addition to event history.

•

We improve the predictive performance of the intensity-

based models through conditionally dependent modeling.

Further, we draw on the intensity-free literature to design

a exible multivariate marked TPP model. We model the

PDF of conditional inter-event time to enable closed-form

likelihood computation and closed-form sampling.

•

Using multiple metrics, we provide a comprehensive evalua-

tion of a diverse set of synthetic and real-world datasets. The

proposed models consistently outperform both conditionally

independent and conditionally dependent models.

2 RELATED WORK

In this section, we provide a brief overview of classical (non-neural)

TPPs and neural TPPs. Later, we discuss conditionally independent

and conditionally dependent models. In the end, we dierentiate the

proposed solution against state-of-the-art models in the literature.

2.1 Classical (non-neural) TPPs

TPPs are mainly described via conditional intensity function. Basic

TPP models make suitable assumptions about the underlying sto-

chastic process resulting in constrained intensity parametrizations.

For instance, Poisson process [

] assumes that inter-event

times are independent. In Hawkes process [

] event excitation

is positive, additive over time, and decays exponentially with time.

Self-correcting process [

] and autoregressive conditional duration

process [

] propose dierent conditional intensity parametrizations

to capture inter-event time dynamics. These constraints on condi-

tional intensity limit the expressive power of the models and hurt

predictive performance due to model misspecication [8].

2.2 Neural TPPs

Neural TPPs are more expressive and computationally ecient

than classical TPPs due to their ability to learn complex dependen-

cies. A TPP model inferring the time and mark of the next event

sequentially is called autoregressive (AR) TPP. A seminal work

by [

] connects the point processes with a neural network by

realizing conditional intensity function using a recurrent neural

network (RNN). Generally, the event history is encoded using either

recurrent encoders or set aggregation encoders [38, 39].

Conditionally independent models assume time and mark are

independent and inferred from the history vector representing

past events. This assumption makes this neural architecture com-

putationally inexpensive but hurts the predictive performance as

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ModelingInter-DependenceBetweenTimeandMarkinMultivariateTemporalPointProcessesGovindWaghmareMastercard,AIGarageGurugram,Indiagovind.waghmare@mastercard.comAnkurDebnathMastercard,AIGarageGurugram,Indiaankur.debnath@mastercard.comSiddharthaAsthanaMastercard,AIGarageGurugram,Indiasiddhartha.asthana@mas...

展开>> 收起<<

Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: