Eye-tracking based classiﬁcation of Mandarin Chinese readers with and without dyslexia using neural sequence models Patrick Haller1 Andreas Säuberli1 Sarah E. Kiener1

2025-05-06 0 0 835.29KB 8 页 10玖币

侵权投诉

Eye-tracking based classiﬁcation of Mandarin Chinese readers with and

without dyslexia using neural sequence models

Patrick Haller1, Andreas Säuberli1, Sarah E. Kiener1

Jinger Pan3, Ming Yan4, Lena A. Jäger1,2

1University of Zurich 2University of Potsdam

3The Education University of Hong Kong 4University of Macau

haller@cl.uzh.ch andreas@cl.uzh.ch sarahelisabeth.kiener@uzh.ch

jpan@eduhk.hk mingyan@um.edu.mo jaeger@cl.uzh.ch

Abstract

Eye movements are known to reﬂect cogni-

tive processes in reading, and psychological

reading research has shown that eye gaze pat-

terns differ between readers with and without

dyslexia. In recent years, researchers have at-

tempted to classify readers with dyslexia based

on their eye movements using Support Vec-

tor Machines (SVMs). However, these ap-

proaches (i) are based on highly aggregated

features averaged over all words read by a par-

ticipant, thus disregarding the sequential na-

ture of the eye movements, and (ii) do not

consider the linguistic stimulus and its inter-

action with the reader’s eye movements. In

the present work, we propose two simple se-

quence models that process eye movements

on the entire stimulus without the need of ag-

gregating features across the sentence. Addi-

tionally, we incorporate the linguistic stimulus

into the model in two ways—contextualized

word embeddings and manually extracted lin-

guistic features. The models are evaluated

on a Mandarin Chinese dataset containing eye

movements from children with and without

dyslexia. Our results show that (i) even for a

logographic script such as Chinese, sequence

models are able to classify dyslexia on eye

gaze sequences, reaching state-of-the-art per-

formance, and (ii) incorporating the linguistic

stimulus does not help to improve classiﬁca-

tion performance.1

1 Introduction

Reading effortlessly constitutes a key skill in mod-

ern society. Individuals suffering from develop-

mental dyslexia are characterized by speciﬁc and

persistent reading problems. Global prevalence es-

timates range from 3 to 7% (Landerl et al.,2013;

Peterson and Pennington,2012). Previous research

has consistently shown that early diagnosis and in-

tervention is key to mitigate the resulting long-term

Model code is publicly available and can be found under

https://github.com/hallerp/dyslexia-seqmod.

Figure 1: Proposed approach. Each eye-movement

reading measure vector is concatenated with contex-

tualized word embeddings and used as input for the

sequence models to infer whether a reader suffers from

dyslexia.

consequences (Vaughn et al.,2010).

Psychological and clinical research on eye move-

ment patterns has revealed that individuals with

dyslexia exhibit gaze patterns that differ signiﬁ-

cantly from the patterns observed in individuals

without dyslexia (Rayner,1998;Pan et al.,2014).

In particular, scanpaths of individuals with dyslexia

are characterized by longer ﬁxation durations, more

ﬁxations, decreased saccade durations and a higher

proportion of regressions. In recent years, increas-

ing effort has been spent on utilizing these ﬁnd-

ings and applying supervised classiﬁcation meth-

ods such as SVMs and Random Forests on eye

movement data (see Kaisar 2020 for an overview)

to infer the presence or absence of dyslexia. There

are several reasons why automatized approaches

for assistance in dyslexia detection are desirable.

Currently, paper-pencil diagnostic tools are con-

ducted by trained speech therapists. These tools

arXiv:2210.09819v2 [cs.CL] 2 Dec 2022

are time-intensive and are typically only consid-

ered after a suspected case has been reported by

observant educational staff, leaving many cases

overlooked. Eye-movement-based diagnostic tools

have the potential to be deployed in schools in a

relatively inexpensive manner and as part of a stan-

dard procedure aimed at early and comprehensive

detection of dyslexia; making an important contri-

bution to educational equity.

Although the aforementioned approaches provide

promising results, they suffer from speciﬁc draw-

backs: (i) The model input consists of eye move-

ment features, aggregated for each subject over the

presented stimulus material (text), thus disregard-

ing the sequential nature of the eye movements; (ii)

both the linguistic stimulus and its interaction with

the reader’s eye movements are not considered. For

classiﬁcation purposes, this does not pose a prob-

lem per se. However, it does not allow us to inves-

tigate questions such as: Which words (or, more

speciﬁcally, what linguistic properties of the stim-

ulus) are particularly informative to discriminate

between individuals with and without dyslexia?

In the present work, we propose two neural se-

quence models, depicted in Figure 1, that process

the eye movements on the entire stimulus without

the necessity of feature aggregation over the sen-

tence. To incorporate the linguistic stimulus into

the model, we use pre-trained contextualized word

embeddings. We evaluate our model on an eye-

tracking-while-reading dataset from children with

and without dyslexia reading Mandarin Chinese

sentences by Pan et al. (2014).

2 Related Work

2.1 ML-based detection of dyslexia

To date, various data types and signals have been

utilized to solve the task of automated detection of

dyslexia such as text, MRI scans (Cui et al.,2016),

EEG recordings (Frid and Breznitz,2012), student

engagement data (Abdul Hamid et al.,2018) as

well as eye-tracking data (Rello and Ballesteros,

2015;Raatikainen et al.,2021;Benfatto et al.,

2016). Benfatto et al. (2016) train a Support Vector

Machine with recursive feature elimination (SVM-

RFE) on 168 eye-tracking features obtained from

an eye-tracking-while-reading dataset from 185

Swedish children (aged 9-10 years). Their best

SVM-RFE model selected 48 features and achieved

an accuracy score of 95.6%

4.5% (sic!) on a bal-

anced dataset. We reimplement this method and

use it as a reference method (cf. 4.1). Jothi Prabha

and Bhargavi (2020), using the same dataset as Ben-

fatto et al. (2016), experiment with various feature

selection algorithms and machine learning mod-

els. They ﬁnd that feature selection via Principle

Component Analysis (PCA) in combination with a

Particle Swarm Optimization based Hybrid Kernel

SVM classiﬁer yields the best accuracy.

Raatikainen et al. (2021) combine a Random For-

est classiﬁer for feature selection with an SVM,

achieving an accuracy of 89.7%. They expand their

feature space with transition matrices that represent

the number of transitions between the different seg-

ments (question, answer selection) in a trial as well

as the number of gaze shifts within one segment.

2.2 Modeling eye-tracking data with deep

neural sequence models

Eye movement data for task inference.

Deep

neural sequence models have been deployed to

solve inference tasks based on eye movements such

as reader (Jäger et al.,2019) and viewer identiﬁ-

cation (Lohr et al.,2020;Makowski et al.,2020,

2021), ADHD detection (Deng et al.,2022) as well

as the prediction of reading comprehension (Reich

et al.,2022).

Integrating the linguistic stimulus.

There has

been growing interest in combining language and

eye movement models to predict gaze patterns dur-

ing naturalistic reading (Hollenstein et al.,2021;

Merkx and Frank,2021;Hollenstein et al.,2022).

Wiechmann et al. (2022) investigate the role of

general text features and their interaction with eye

movement patterns in predicting human reading

behavior and ﬁnd that models incorporating the

linguistic stimulus improves prediction accuracy.

3 Problem Setting

We investigate the two closely related tasks of

classifying (i) whether a given eye gaze sequence

on one sentence is from a reader with or with-

out dyslexia and (ii) whether a given eye gaze

sequence on a set of sentences is from a reader

with or without dyslexia. Formally, our train-

ing data can be represented as a set

{(W11, y1),...,(WNM , yN)}

, where

Wij =

hwij1. . . wijK i

is a sequence of reading measure

vectors

for each word

k∈1. . . Kj

obtained from

subject

reading sentence

, where

is the num-

ber of participants,

is the number of stimulus

2Cf. the list of reading measures in Appendix B.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Eye-trackingbasedclassicationofMandarinChinesereaderswithandwithoutdyslexiausingneuralsequencemodelsPatrickHaller1,AndreasSäuberli1,SarahE.Kiener1JingerPan3,MingYan4,LenaA.Jäger1;21UniversityofZurich2UniversityofPotsdam3TheEducationUniversityofHongKong4UniversityofMacauhaller@cl.uzh.chandreas@cl.uz...

展开>> 收起<<

Eye-tracking based classiﬁcation of Mandarin Chinese readers with and without dyslexia using neural sequence models Patrick Haller1 Andreas Säuberli1 Sarah E. Kiener1.pdf

共8页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Eye-tracking based classiﬁcation of Mandarin Chinese readers with and without dyslexia using neural sequence models Patrick Haller1 Andreas Säuberli1 Sarah E. Kiener1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: