How do you go where Improving next location prediction by learning travel mode information using transformers_2

2025-04-29 0 0 1.5MB 10 页 10玖币

侵权投诉

How do you go where? Improving next location prediction by

learning travel mode information using transformers

Ye Hong

hongy@ethz.ch

Institute of Cartography and

Geoinformation, ETH Zurich

Zurich, Switzerland

Henry Martin

martinhe@ethz.ch

Institute of Cartography and

Geoinformation, ETH Zurich

Zurich, Switzerland

Institute of Advanced Research in

Articial Intelligence (IARAI)

Vienna, Austria

Martin Raubal

mraubal@ethz.ch

Institute of Cartography and

Geoinformation, ETH Zurich

Zurich, Switzerland

ABSTRACT

Predicting the next visited location of an individual is a key problem

in human mobility analysis, as it is required for the personaliza-

tion and optimization of sustainable transport options. Here, we

propose a transformer decoder-based neural network to predict

the next location an individual will visit based on historical lo-

cations, time, and travel modes, which are behaviour dimensions

often overlooked in previous work. In particular, the prediction

of the next travel mode is designed as an auxiliary task to help

guide the network’s learning. For evaluation, we apply this ap-

proach to two large-scale and long-term GPS tracking datasets

involving more than 600 individuals. Our experiments show that

the proposed method signicantly outperforms other state-of-the-

art next location prediction methods by a large margin (8

05% and

60% relative increase in F1-score for the two datasets, respec-

tively). We conduct an extensive ablation study that quanties the

inuence of considering temporal features, travel mode informa-

tion, and the auxiliary task on the prediction results. Moreover,

we experimentally determine the performance upper bound when

including the next mode prediction in our model. Finally, our anal-

ysis indicates that the performance of location prediction varies

signicantly with the chosen next travel mode by the individual.

These results show potential for a more systematic consideration

of additional dimensions of travel behaviour in human mobility

prediction tasks. The source code of our model and experiments is

available at https://github.com/mie-lab/location-mode-prediction.

CCS CONCEPTS

•Information systems →Geographic information systems

;

Location based services;

•Computing methodologies →Neural

networks;•Applied computing →Transportation.

KEYWORDS

Mobility, Deep learning, Location prediction, Travel behaviour

1 INTRODUCTION

The rapid urbanization process in the last decades has caused a con-

stant increase in individual travel, imposing signicant challenges

in achieving sustainable cities. To meet the sustainable develop-

ment goals of the United Nations [

], mobility behaviour change

and new mobility concepts to promote these changes will play in-

dispensable roles [

]. These mobility concepts, such as mobility as

a service (MaaS) [

], smart charging [

] and ride-sharing [

], all

rely on the capability to proactively provide personalized services

that are tailored to the travel context and individuals’ characteris-

tics [

]. Individual mobility prediction to know when and where

travel will occur is a crucial technique driving the development and

application of these new concepts and, therefore, a key technology

for sustainable transportation.

The prediction of where an individual will go given her historical

mobility information is central to individual mobility prediction.

The problem is also known as the next location prediction and has

attracted much attention over the last decade. Researchers are in-

creasingly interested in tackling this problem using learning-based

methods thanks to the booming of deep learning (DL) models [

As it can be formulated as a sequence prediction problem, similar

to the tasks encountered in natural language processing and audio

processing, models that have shown success in these two elds are

often directly applied. In particular, the transformer model [

] that

utilizes a multi-head self-attention mechanism has revolutionized

various sequence modelling tasks due to its powerful and ecient

network structure. Transformer models are also starting to gain

attention in predicting individual mobility, as they tackle some chal-

lenges in mobility prediction by design: (1) Multiple periodicities

co-exist in the location visitation patterns of individuals [

] (e.g.,

daily, weekly). These periodicities vary considerably across individ-

uals. The multi-head self-attention module allows the network to

focus on multiple steps in the input sequence, eectively capturing

these periodicities. (2) The long-term dependency of mobility be-

haviour. Studies have shown that the current mobility depends on

behaviours conducted days or weeks before [

], which requires

the prediction model to capture long-term dependencies. The de-

sign of the transformer model enables ecient learning of these

dependencies. However, human mobility also exhibits unique char-

acteristics, such as complex spatio-temporal dependencies [

]

and the inherent stochasticity of location visits [

], which hinder

the performance when applying sequence learning models on raw

location visit sequences. Therefore, learning to predict the next

location directly from historical location visits is challenging. An

accurate prediction model should consider context information that

inuences individuals’ choice of locations.

Results from travel behaviour studies that aim to understand

individuals’ activity location choices could guide the consideration

of context information. Empirical evidence suggests that the se-

lection of activity locations is highly correlated to other aspects

arXiv:2210.04095v2 [cs.LG] 27 Oct 2022

Ye Hong, Henry Martin, and Martin Raubal

of individual travel behaviour, such as the availability of travel

modes [

] and the day of the week [

]. However, the comprehen-

sive information regarding individuals’ travel behaviour is not fully

utilized in location prediction problems. To date, it is still unclear (1)

how strong the inuence of these long-term factors is on choosing

the immediate next location and (2) whether the DL network can

benet from this knowledge and learn the complex dependency

patterns directly from data.

To close this research gap and answer the above questions, we

propose a transformer-based model that utilizes historical travel

behaviour to predict individuals’ next location. More precisely, the

model aims to learn mobility transition patterns from historical

location, temporal and travel mode sequence information. Inspired

by travel behaviour studies, we encourage the model to also pre-

dict the next travel mode the individual will choose. We anticipate

that this ancillary task will help the prediction of the next loca-

tion. Through experiments on two real-world GPS datasets, we

demonstrate the eectiveness of our model design and quantify

the dependency of location prediction performance on travel mode.

Our results show that careful consideration of individual travel be-

haviour signicantly benets human mobility prediction. In short,

our contributions are summarized as follows:

•

We propose a transformer decoder-based neural network

that utilizes location, travel mode and time-related infor-

mation for the next location prediction task. The proposed

model achieves state-of-the-art performance.

•

We show that jointly learning the next location and next

mode improves the prediction performance for both tasks.

•

We conduct extensive experiments on two real-world GPS

tracking datasets and conclude that considering additional

aspects of travel behaviour signicantly increases the per-

formance of next location prediction.

The rest of this paper is organized as follows. We rst systemati-

cally review related work in Section 2. In Section 3, we formulate

the next location prediction problem. Next, we introduce details of

the network architecture in Section 4. We apply our model to two

real-world GPS datasets and analyze its performance in Section 5.

Finally, we summarize the main ndings and conclude the paper in

Section 6.

2 RELATED WORK

2.1 Next location prediction

The next location prediction problem has found application in many

dierent elds, such as recommendation systems [

], sensor net-

works [

], and mobility behaviour analysis [

]. The exact

denition of the problem varies across studies due to dierent ob-

jectives and employed datasets. For example, location-based social

network (LBSN) applications focus on predicting the next check-

in point-of-interest (POI) [

]. In contrast, mobility behaviour

studies aim to understand the next location for a user to conduct an

activity [

]. Here we focus on the methods proposed for mobility

applications.

The last decade has witnessed the expansion of studies focus-

ing on next location prediction. Markov Chain and its variants are

probably the most often employed methods for the task [

]. These

models regard locations as states and construct a transition ma-

trix that encodes the transition probability between states for each

individual. Ashbrook and Starner

[1]

and Gambs et al

. [11]

both

proposed identifying signicant locations from GPS data and build-

ing a Markov model to predict location transitions. Later Markov

model variants that consider collective movements [

] and incor-

porate location importance [

] further increased the prediction

performance. However, Markov-based models struggle to represent

the complex sequential patterns in human mobility because of their

inherent assumption that the current state only depends on the

states of previously limited time steps [20].

Recent advances in DL have also promoted their application

in location prediction. As a widely adopted sequence modelling

method, recurrent neural network (RNN)-based models, such as

Long Short-Term Memory (LSTM) [

] and spatial-temporal (ST)-

RNN [

], were reported to outperform Markov models by a large

margin in the task. Still, vanilla RNN models tend to underweight

long-term dependencies when the input sequence length increases.

Therefore, studies employed the attention module to capture both

short-term and long-term dependencies dynamically [

]. More-

over, the transformer model that builds on top of the multi-head self-

attention mechanism [

] have started to gain interest in the eld.

In particular, Xue et al

. [44]

proposed MobTcast for considering

various contexts with a transformer-based structure and achieved

state-of-the-art POI prediction results for LBSN data. Although

having great potential in learning the complex spatio-temporal

dependencies, limited studies have applied transformer for the lo-

cation prediction problem.

2.2 Factors aecting activity location choice

Understanding the factors aecting activity location choice is bene-

cial for predicting individuals’ mobility, as they can be regarded as

prior knowledge and potentially guide the learning of DL models. In

the travel behaviour eld, the choice of locations is regarded as an

integral part of individuals’ activity-travel behaviour and has been

studied within the activity-based framework [

]. Studies that focus

on analysing travel behaviour over time suggest that both stability

and variability are found in individuals’ activity location choices.

For example, Dharmowijoyo et al

. [6]

showed that the variability

of location visits is much larger between weekend-weekday pairs

than between weekday-weekday and weekend-weekend pairs. Em-

pirical studies also demonstrate the correlation of dierent aspects

of individual travel behaviour. For example, Susilo and Axhausen

[38]

reported high repetition in location-mode combinations, sug-

gesting that individuals use the same travel mode to reach their

locations. Similar conclusions were reported by Hong et al

. [14]

where they found that only a subset of all location-mode combi-

nations is essential for describing the mobility behaviour. From

this perspective, aspects of travel behaviour can be considered

constraints for individuals’ choice of activity locations.

A similar problem as the next location prediction is the for-

mulation of an individual’s location choice set, which is a crucial

component in microscopic trac simulation models [

]. Instead of

predicting the exact next location, the problem aims at generating

a set containing all possible locations. Based on time geography

theory, potential path areas analysis has been applied to tackle

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Howdoyougowhere?ImprovingnextlocationpredictionbylearningtravelmodeinformationusingtransformersYeHonghongy@ethz.chInstituteofCartographyandGeoinformation,ETHZurichZurich,SwitzerlandHenryMartinmartinhe@ethz.chInstituteofCartographyandGeoinformation,ETHZurichZurich,SwitzerlandInstituteofAdvancedResear...

展开>> 收起<<

How do you go where Improving next location prediction by learning travel mode information using transformers_2.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

How do you go where Improving next location prediction by learning travel mode information using transformers_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: