A Transformer-based deep neural network model for SSVEP classification_2

2025-04-27 0 0 536.43KB 33 页 10玖币

侵权投诉

arXiv:2210.04172v1 [q-bio.NC] 9 Oct 2022

A Transformer-based deep neural network model for

SSVEP classiﬁcation

Jianbo Chena, Yangsong Zhanga,∗

, Yudong Pana, Peng Xub,∗

, Cuntai Guanc

aLaboratory for Brain Science and Medical Artiﬁcial Intelligence, School of Computer

Science and Technology, Southwest University of Science and Technology, Mianyang, China

bMOE Key Laboratory for NeuroInformation, Clinical Hospital of Chengdu Brain Science

Institute, and Center for Information in BioMedicine, School of Life Science and

Technology, University of Electronic Science and Technology of China, Chengdu, China

cSchool of Computer Science and Engineering, Nanyang Technological University, Singapore

Abstract

Steady-state visual evoked potential (SSVEP) is one of the most commonly used

control signal in the brain-computer interface (BCI) systems. However, the con-

ventional spatial ﬁltering methods for SSVEP classiﬁcation highly depend on

the subject-speciﬁc calibration data. The need for the methods that can al-

leviate the demand for the calibration data become urgent. In recent years,

developing the methods that can work in inter-subject classiﬁcation scenario

has become a promising new direction. As the popular deep learning model

nowadays, Transformer has excellent performance and has been used in EEG

signal classiﬁcation tasks. Therefore, in this study, we propose a deep learning

model for SSVEP classiﬁcation based on Transformer structure in inter-subject

classiﬁcation scenario, termed as SSVEPformer, which is the ﬁrst application of

the transformer to the classiﬁcation of SSVEP. Inspired by previous studies, the

model adopts the frequency spectrum of SSVEP data as input, and explores the

spectral and spatial domain information for classiﬁcation. Furthermore, to fully

utilize the harmonic information, an extended SSVEPformer based on the ﬁlter

bank technology (FB-SSVEPformer) is proposed to further improve the clas-

siﬁcation performance. Experiments were conducted using two open datasets

∗Corresponding authors: Yangsong Zhang(zhangysacademy@gmail.com); Peng

Xu(xupeng@uestc.edu.cn)

Preprint submitted to Neural Networks October 11, 2022

(Dataset 1: 10 subjects, 12-class task; Dataset 2: 35 subjects, 40-class task)

in the inter-subject classiﬁcation scenario. The experimental results show that

the proposed models could achieve better results in terms of classiﬁcation ac-

curacy and information transfer rate, compared with other baseline methods.

The proposed model validates the feasibility of deep learning models based on

Transformer structure for SSVEP classiﬁcation task, and could serve as a po-

tential model to alleviate the calibration procedure in the practical application

of SSVEP-based BCI systems.

Keywords: Brain-computer interface, Steady-state visual evoked potential,

Transformer, Deep learning, Filter bank

1. Introduction

Brain-computer interface (BCI) has become a popular research direction

in human-computer interaction and medical rehabilitation, which can directly

connect the brain to external devices without going through the peripheral

nervous system, enabling bidirectional information transmission and feedback

[42, 47]. Electroencephalogram (EEG)-based BCIs obtain the intentions of the

brain through EEG signals, and have attracted attention due to the advantages

of convenience, low cost, and non-invasiveness [1]. Among the various EEG

paradigms, the high signal-to-noise ratio and low training time of steady-state

visual evoked potential (SSVEP) make it one of the most popular paradigms.

SSVEP refers to the EEG in the visual cortex when the subject gazes at a

ﬂickering visual stimulus modulated by a constant frequency [56]. The frequen-

cies of SSVEP are the same as the coding frequency of received visual stimuli

as well as its harmnoics[33]. By virtue of this characteristic of SSVEP, it is

possible to design SSVEP-based BCI system, such as SSVEP-based speller [27],

in which diﬀerent targets are encoded by diﬀerent stimulus frequencies. When

the subjects need to select a command, they can gaze at the corresponding ﬂick-

ering target stimulus that coding the command on the interface. The generated

SSVEP can be identiﬁed by a specially designed decoder to obtain the intention

of the subject.

In the SSVEP-based BCI system, the robust classiﬁcation of the SSVEPs

is very important [28]. As the SSVEP frequency is the same as the stimulus

frequency, some researches developed the algorithms based on the prior fre-

quency information, such as power spectral density analysis (PSDA) [40] and

canonical correlation analysis (CCA) [22], etc. In addition to the fundamen-

tal frequency component, SSVEP also contains harmonic components whose

frequencies are multiples of the fundamental frequency [25]. Based on this char-

acteristic, ﬁlter bank technology was introduced to extend the original CCA

(FBCCA) [5]. FBCCA uses CCA in multiple subbands of SSVEP data, and

ﬁnally weights the correlation coeﬃcients calculated from these subbands. The

FBCCA improves the classiﬁcation performance by distinguishing the funda-

mental frequency and harmonics, demonstrating the eﬀectiveness of the ﬁlter

bank technique on SSVEP classiﬁcation. Nowadays, ﬁlter bank technology has

been widely used in various methods [57, 31].

However, due to the complexity of EEG, SSVEP data always contains noises,

such as spontaneous EEG and electromagnetic interference, seriously polluting

the signal [17]. Traditional training-free methods (such as PSDA, CCA) have

better results only when the data length is long. To address the noise inter-

ference in SSVEP, a series of recognition algorithms based on machine learning

have been proposed. Such methods perform under the intra-subject classiﬁ-

cation condition, in which the training and testing data are from the same

subjects, as shown in Fig. 1(a). In this condition, the model can obtain pa-

rameters that are more suitable for a speciﬁc subject, thereby reducing noise

interference [50]. For example, individual template based CCA (IT-CCA) cal-

culates the average of the subject’s existing SSVEP signals at each stimulation

frequency and uses it as the reference signal for CCA [4]. This method can add

subject-speciﬁc patterns to the reference signal, and is widely used in subsequent

algorithms. Task-related component analysis (TRCA) method obtains spatial

ﬁlters by maximizing the reconstitution between SSVEPs of diﬀerent trials to

reduce the noise of SSVEPs and reference signals [26]. Correlated component

Training data

New subjects

Training Testing

Classifier

瀥瀥瀥

Testing data

Existing subjects

瀥瀥瀥

Training data

Training Testing

Classifier

Testing data

(a)

(b)

Figure 1: The diagram of two classiﬁcation scenarios. (a) intra-subject classiﬁcation; (b)

inter-subject classiﬁcation.

analysis (CORCA) learns spatial ﬁlters by maximizing the correlation between

data to reduce background noise [55]. Task-discriminant component analysis

(TDCA) uses multi-class linear discriminant analysis to learn spatiotemporal

ﬁlters and classify in a discriminant manner [23].

The above method has signiﬁcant eﬀect in the intra-subject classiﬁcation

experiment, in which the training data and the testing data belong to the same

subject [32]. However, the collection of SSVEP data is a time-consuming and

laborious work. Hence, a potential and challenging direction is to transfer the

data from existing subjects to new subjects in the inter-subject classiﬁcation

scenario, under which a classiﬁer can be obtained with the data from already

existing subjects and then used the classiﬁer to test the data from new subjects,

as shown in Fig. 1(b). Although many works have improved traditional state-of-

the-art methods to adapt them to inter-subject scenario, the results may be not

optimal [46]. Because the brain processes natural sensory stimuli in a dynamic,

non-ﬁxed, and nonlinear manner, SSVEP is non-stationary and varies widely

among individuals [18]. Even the data collected by the same subject, the data

acquired at diﬀerent times may also have diﬀerent distribution. These situa-

tions pose great challenges for inter-subject experiments, and the performance

of traditional machine learning-based algorithms under inter-subject condition

degrades greatly, which is far from its performance under intra-subject condi-

tion.

In recent years, deep learning has been developed signiﬁcantly and has made

milestone progress in areas such as computer vision and natural language pro-

cessing [19, 8]. Deep learning models have powerful feature extraction capabil-

ities and can directly be applied on the raw data[9, 34]. Deep learning models

have been used on many EEG classiﬁcation tasks, including convolutional neu-

ral networks (CNN) [60], recurrent neural networks (RNN) [12], graph neural

networks (GNN) [59], etc. Several studies have used deep learning to process

SSVEP data, achieving outstanding performance on classiﬁcation tasks, espe-

cially inter-subject classiﬁcation. For instances, EEGNet is a compact convolu-

tional neural network (CNN) that uses CNNs to implement the spatial-temporal

ﬁltering and feature extraction, achieving signiﬁcantly better results than tra-

ditional methods under inter-subject conditions [41]. The idea of using tem-

poral and spatial convolutions has achieved promising results, which has also

inﬂuenced many later algorithms. The subject invariant SSVEP generative ad-

versarial network (SIS-GAN) uses generative adversarial networks to generate

artiﬁcial SSVEP data to expand the training dataset [2]. Complex convolu-

tional neural network (CCNN) uses the complex spectrum of SSVEP signal

as the input of CNN for classiﬁcation, demonstrating the eﬀectiveness of com-

plex spectral features on SSVEP classiﬁcation [32]. InceptionEEG-Net (IENet)

combines Inception with residual connections and uses multi-scale convolution

kernels to extract features from receptive ﬁelds of diﬀerent sizes [13]. In addi-

tion, ﬁlter bank technology is also applied in deep learning models to extend

the existing models, such as FB-EEGNet and FBCNN [48, 58].

Although the deep learning-based SSVEP recognition model has made great

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

arXiv:2210.04172v1[q-bio.NC]9Oct2022ATransformer-baseddeepneuralnetworkmodelforSSVEPclassiﬁcationJianboChena,YangsongZhanga,∗,YudongPana,PengXub,∗,CuntaiGuancaLaboratoryforBrainScienceandMedicalArtiﬁcialIntelligence,SchoolofComputerScienceandTechnology,SouthwestUniversityofScienceandTechnology,Miany...

展开>> 收起<<

A Transformer-based deep neural network model for SSVEP classification_2.pdf

共33页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

A Transformer-based deep neural network model for SSVEP classification_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: