Slippage-robust Gaze Tracking for Near-eye Display Wei Zhang1 Jiaxi Cao1 Xiang Wang2 Enqi Tian1and Bin Li1 Abstract In recent years head-mounted near-eye display

2025-05-03 0 0 2.44MB 7 页 10玖币

侵权投诉

Slippage-robust Gaze Tracking for Near-eye Display

Wei Zhang1, Jiaxi Cao1, Xiang Wang2, Enqi Tian1and Bin Li1

Abstract— In recent years, head-mounted near-eye display

devices have become the key hardware foundation for virtual

reality and augmented reality. Thus head-mounted gaze track-

ing technology has received attention as an essential part of

human-computer interaction. However, unavoidable slippage of

head-mounted devices (HMD) often results higher gaze tracking

errors and hinders the practical usage of HMD. To tackle

this problem, we propose a slippage-robust gaze tracking for

near-eye display method based on the aspheric eyeball model

and accurately compute the eyeball optical axis and rotation

center. We tested several methods on datasets with slippage

and the experimental results show that the proposed method

signiﬁcantly outperforms the previous method (almost double

the suboptimal method).

I. INTRODUCTION

With the prosperity of the metaverse, the demand for

augmented, virtual and mixed reality (AR, VR, and MR)

is rapidly increasing. Head-mounted near-eye display, func-

tioning as the hardware foundation of all three techniques,

project virtual scenes onto the human eye and create an

immersive environment. Although various interaction meth-

ods, including gripping and keyboard, are explored [1],

[2], eye gaze movement is still the most natural interactive

way in HMD. Thus, head-mounted near-eye gaze tracking

technology shows excellent potential in human-computer

interaction.

Based on knowledge of human gaze mode in the fun-

damental environment, gaze-based improvements in graphic

rendering and displaying [3], [4], [5] have been studied.

Furthermore, the human gaze directly indicates human per-

ceptions and attention, which is suitable for human-computer

interaction. Gaze cues have already been adopted in in-

teractions between human and intelligent systems [6], [7],

[8] and similar interaction studies have been conducted on

AR/VR devices with gaze tracking implementation. Results

show that gaze-based interaction promises higher interaction

quality, speed, and a more user-friendly experience [9].

Unfortunately, in common AR/VR application scenarios,

body motions and head motions constantly exist during

practical use, which leads to an unavoidable HMD slippage

and slight slippage often results in a signiﬁcant increase in

gaze estimation error of head-mounted devices [10], [11].

Most commercial eye trackers also suffer from the accuracy

degradation caused by slippage [12]. Methods to maintain

gaze accuracy after slippage have been investigated [13] and

1School of Information Science, University of Science and Technology

of China, HeFei 230026, China. zw1996@mail.ustc.edu.cn

2Department of Electrical and Computer Engineering, Carnegie Mellon

University, Pittsburgh, USA. xiangw2@andrew.cmu.edu

Corresponding author:Bin Li (binli@ustc.edu.cn)

results show the robustness of headset slippage in head-

mounted gaze tracking. However, the overall gaze estimation

accuracy of slippage-robust methods is lower than most state-

of-the-art methods and only qualiﬁed for limited uses.

In this work, we proposed an eyeball optical axis and

position estimation method based on the aspheric eye model.

Then we proposed a gaze tracking geometric model for the

near-eye display to estimate scene gaze points. Specially,

we set up a low-cost hardware device to validate our slip-

page robust gaze tracking method for near-eye display. We

conducted experiments on nine subjects and asked them

to remount and rotate the device repeatedly to simulate

slippage in actual use. The experiment results show that our

method outperforms the state-of-art method [13] by 100%,

the angular offset decreases from 1.51◦to 0.76◦.

II. RELATED WORK

Usually, gaze point tracking includes two branches: the

data-driven method and the model drive method. The data-

driven method directly maps the eyeball image into the gaze

point, while the model-driven method relies on the eyeball

model and related optical knowledge. A widely used eye

model is the Le Grand model [14], shown in Figure 1. Le

Grand model views the cornea as a sphere and ignores the

corneal refraction. It is perfect for remote gaze tracking [15],

[16], [17] since the approximations in the model do not

introduce signiﬁcant errors in eyeball detection. However,

in the near-display device, such an error is non-neglectable.

The eyeball in head-mounted gaze tracking must be in the

correct position to achieve the ideal visual appearance and

angle. False eye position might cause image distortion in

VR [18] and object misalignment in MR.

POptical Axis

Visual Axis

Kappa Angle

Fig. 1. Simpliﬁed eyeball model. Optical axis contains pupil center P,

cornea center Cand eyeball rotation center E. The visual axis intersects

with the optical axis at the cornea center, and the angle between them

deﬁnes as the kappa angle.

A more precise eyeball model is required in HMD

gaze point tracking to mend the modeling error. For ex-

arXiv:2210.11637v2 [cs.CV] 1 Nov 2022

ample, Nitschke proposed a new eye position estimation

method [19]. Y. Itoh proposed an interaction-free AR cal-

ibration method [20] for eye position acquisition. However,

predeﬁned eyeball parameters introduce deviation among

people. Estimating eyeball rotation center with eye poses was

discussed in gaze tracking [21] as well, but the result was

highly dependent on pupil contour tracking. In addition, there

have been studies of pupil center[22] and corneal center[23],

[24]. A robust eyeball position estimation for various persons

remains a problem in head-mounted systems.

Head-mounted gaze estimation is the basis of gaze-

contingent near-eye display realization. A defect of most

existing head-mounted gaze tracking methods is the lack of

slippage robustness. During common use, a slight relative

movement between the head-mounted device and the eyes

is unavoidable. Keeping high gaze estimation accuracy with

slippage taken into consideration should be realized in prac-

tical head-mounted systems.

Model-based gaze tracking methods [25] are often con-

sidered robust to devise slippage, but complicated system

calibration and relatively low accuracy made these methods

less famous. The state-of-the-art feature-based gaze estima-

tion method proposed by PupilLab [26] achieved a mean

angular error lower than 0.6◦. A combination of PupilLab

eye tracker and VR systems has been studied [11], [27].

However, the method purely relied on the 2D pupil fea-

ture and was highly sensitive to slippage. The commonly

accepted slippage-robust gaze tracking method Grip [13]

used a model-based feature (optical axis) and a feature-based

gaze mapping model. An acceptable gaze accuracy with

slippage robustness was shown. End-to-end learning-based

gaze estimation method [28] was proved to be robust and

accurate when headset slippage occurred, but model training

was data-consuming and required users to collect a large

amount of calibration data.

Recently, slippage in head-mounted gaze tracking has

become a critical research problem, and increasing works

have been aware of the importance of slippage robustness.

Still, few works discussed slippage-robust near-eye display

gaze tracking.

III. SYSTEM OVERVIEW

This section presents our gaze tracking for near-eye dis-

play system, where we build hardware devices that can

capture eye images, extract features, and have an optical see-

through display. Figure 2 illustrates the workﬂow of slippage

robust gaze tracking for near-eye display.

A. Hardware Setup

Our headset prototype consists of a near-eye display, 4 eye

cameras capturing eye images of 640×480pix2, and 850 nm

infrared LEDs.

Based on our proposed eye model, determining the eyeball

optical axis requires a virtual pupil center and a corneal

surface normal vector containing eye camera center. The Vir-

tual pupil centers can be captured in eye images. A Corneal

surface normal vector containing eye camera center can be

(a) (b)

Fig. 2. Overview of the proposed system. (a) Hardware prototype of

proposed near-eye display system implemented with an eye tracker. (b) For

each eye, there are two near-focus infrared cameras to capture eye images,

and six infrared LEDs around each eye camera are used to generate corneal

reﬂection glints. A binocular optical see-through display is implemented in

the prototype as a near-eye display. (c) The subject was asked to gaze at the

front vehicle. Pupil centers and optical axes detection results were displayed

in eye images. (d) The subject can move and rotate the HMD within a certain

range. Results of gaze tracking remained highly accurate. The gaze point

estimation result was visualized as red circles in the displayed image. The

scene image for gaze estimation demonstration came from svl-simulator

(https://www.svlsimulator.com/).

derived by the corneal reﬂection of a light source coincides

with the eye camera center [29]. However, implementing a

light source inside eye camera is not physically realizable.

In our prototype, 6 infrared LEDs around the eye camera are

used to generate corneal reﬂection glints. The centroid of a

group of glints approximately provides the reﬂection position

of a light source coincides with the eye camera center.

A binocular optical see-through display is implemented in

the prototype as a near-eye display. Both monocular displays

show images of 1920×1080pix2, and the binocular ﬁeld of

view is 44◦.

B. Feature Detection

Raw feature detection in eye images includes glint detec-

tion and pupil detection.

Glints appear to be several small bright regions with

special patterns in the eye images. A simple thresholding

and morphology method will give a centroid result of glints.

A problem observed from collected data is that contact lenses

might cause glint distortion. The glints reﬂected by the edge

of the contact lens or the unsmooth surface of the contact

lens might lead to errors in glint detection.

Pupil detection is a mature technique in gaze tracking

technology. We employ radial symmetry transform [30] to

detect coarse pupil center, the starburst algorithm [31] to

detect pupil edges, PURE [32] selects pupil edges and ﬁt

pupil ellipse. Since eye images are captured in near infrared

band, environment illumination produces less effect on image

quality and pupil detection remains accurate. Several exem-

plars of pupil detection results are given in Figure 3.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Slippage-robustGazeTrackingforNear-eyeDisplayWeiZhang1,JiaxiCao1,XiangWang2,EnqiTian1andBinLi1AbstractInrecentyears,head-mountednear-eyedisplaydeviceshavebecomethekeyhardwarefoundationforvirtualrealityandaugmentedreality.Thushead-mountedgazetrack-ingtechnologyhasreceivedattentionasanessentialpartof...

展开>> 收起<<

Slippage-robust Gaze Tracking for Near-eye Display Wei Zhang1 Jiaxi Cao1 Xiang Wang2 Enqi Tian1and Bin Li1 Abstract In recent years head-mounted near-eye display.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Slippage-robust Gaze Tracking for Near-eye Display Wei Zhang1 Jiaxi Cao1 Xiang Wang2 Enqi Tian1and Bin Li1 Abstract In recent years head-mounted near-eye display

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: