Real Spike Learning Real-valued Spikes for Spiking Neural Networks Yufei Guo Liwen Zhang Yuanpei Chen Xinyi Tong Xiaode Liu YingLei

2025-04-24 0 0 1.75MB 18 页 10玖币

侵权投诉

Real Spike: Learning Real-valued Spikes for

Spiking Neural Networks

Yufei Guo⋆, Liwen Zhang⋆, Yuanpei Chen, Xinyi Tong, Xiaode Liu, YingLei

Wang, Xuhui Huang , and Zhe Ma

Intelligent Science & Technology Academy of CASIC, Beijing 100854, China

yfguo@pku.edu.cn, lwzhang9161@126.com, starhxh@126.com, mazhe thu@163.com

Abstract. Brain-inspired spiking neural networks (SNNs) have recently

drawn more and more attention due to their event-driven and energy-

eﬃcient characteristics. The integration of storage and computation

para-digm on neuromorphic hardwares makes SNNs much diﬀerent from

Deep Neural Networks (DNNs). In this paper, we argue that SNNs may

not beneﬁt from the weight-sharing mechanism, which can eﬀectively

reduce parameters and improve inference eﬃciency in DNNs, in some

hardwares, and assume that an SNN with unshared convolution kernels

could perform better. Motivated by this assumption, a training-inference

decoupling method for SNNs named as Real Spike is proposed, which

not only enjoys both unshared convolution kernels and binary spikes in

inference-time but also maintains both shared convolution kernels and

Real-valued Spikes during training. This decoupling mechanism of SNN

is realized by a re-parameterization technique. Furthermore, based on the

training-inference-decoupled idea, a series of diﬀerent forms for imple-

menting Real Spike on diﬀerent levels are presented, which also enjoy

shared convolutions in the inference and are friendly to both neuromor-

phic and non-neuromorphic hardware platforms. A theoretical proof is

given to clarify that the Real Spike-based SNN network is superior to its

vanilla counterpart. Experimental results show that all diﬀerent Real

Spike versions can consistently improve the SNN performance. More-

over, the proposed method outperforms the state-of-the-art models on

both non-spiking static and neuromorphic datasets.

1 Introduction

Spiking Neural Networks (SNNs) have received increasing attention as a novel

brain-inspired computing model that adopts binary spike signals to communi-

cate between units. Diﬀerent from the Deep Neural Networks (DNNs), SNNs

transmit information by spike events, and the computation dominated by the

addition operation occurs only when the unit receives spike events. Beneﬁtting

from this characteristic, SNNs can greatly save energy and run eﬃciently when

implementing on neuromorphic hardwares, e.g., SpiNNaker [17], TrueNorth [1],

Darwin [28], Tianjic [32], and Loihi [5].

⋆Equal contribution.

arXiv:2210.06686v1 [cs.NE] 13 Oct 2022

2 Guo, Y. et al.

* =

The Convolution of SNNs Implemented on Neuromorphic Hardwares

Input Map Convolution Kernel Output Map

Convolution

Kernel Vector Input Map Matrix

Output Map Vector

Input Map

Neurons

Convolution Kernel

Connections

Ouput Map

Neurons

A Neuromorphic

Hardware

The Convolution of DNNs Implemented on Deep Learning hardwares

The Convolution

Fig. 1. The diﬀerence of convolution computing process between DNNs and SNNs. For

DNNs, the calculation is conducted in a highly-paralleled way on conventional hard-

wares. However, for SNNs, each connection between neurons will be mapped into a

synapse on some neuromorphic hardwares, which cannot beneﬁt from the advantages

of the weight-shared convolution kernel, e.g., inference acceleration and parameter re-

duction.

The success of DNNs inspires the SNNs in many ways. Nonetheless, the rich

spatio-temporal dynamics, event-driven paradigm, and friendly to neuromor-

phic hardwares make SNNs much diﬀerent from DNNs, and directly applying

the successful experience of DNNs to SNNs may limit the performance of SNNs.

As one of the most widely used techniques in DNNs, the weight-shared con-

volution kernel shows great advantages. It can reduce the parameters of the

network and accelerate the inference. However, SNNs show great advantages in

the condition of being implemented on neuromorphic hardwares which is very

diﬀerent from DNNs being implemented on deep learning hardwares. As shown

in Fig. 1, in DNNs, the calculation is carried out in a highly-paralleled way on

deep learning hardwares, thus sharing convolution kernel can improve the com-

puting eﬃciency by reducing the data transferring between separated memory

and processing units. However, for an ideal situation, to take full advantage of

the storage-computation-integrated paradigm of neuromorphic hardwares, each

unit and connection of the SNNs in the inference phase should be mapped into

a neuron and synapse in neuromorphic hardware, respectively. Though these

hardwares could be multiplexed, it also increases the complexity of deploymen-

tion and extra cost of data transfer. As far as we know, at least Darwin [28],

Tianjic [32], and other memristor-enabled neuromorphic computing systems [41]

adopt this one-to-one mapping form at present. Hence, all the components of

an SNN will be deployed as a ﬁxed conﬁguration on these hardwares, no matter

Real Spike 3

they share the same convolution kernel or not. Unlike the DNNs, the shared con-

volution kernels will not bring SNNs the advantages of parameter reduction and

inference acceleration in this situation. Hence we argue that it would be better

to learn unshared convolution kernels for each output feature map in SNNs.

Unfortunately, whether in theory or technology, it is not feasible to directly

train an unshared convolution kernels-based SNN. First, there is no obvious proof

that learning diﬀerent convolution kernels directly will surely beneﬁt the network

performance. Second, due to the lack of mature development platforms for SNNs,

many eﬀorts are focusing on training SNNs with DNN-oriented programming

frameworks, which usually do not support the learning of unshared convolution

kernels for each feature map directly. Considering these limitations, we focus on

training SNNs with unshared convolution kernels based on the modern DNN-

oriented frameworks indirectly.

Driven by the above reasons, a training-time and inference-time decoupled

SNN is proposed, where a neuron can emit real-valued spikes during training

but binary spikes during inference, dubbed Real Spike. The training-time real-

valued spikes can be converted to inference-time binary spikes via convolution

kernel re-parameterization and a shared convolution kernel, which can be de-

rived into multiples then (see details in Sec. 3.3). In this way, an SNN with

diﬀerent convolution kernels for every output feature map can be obtained as

we expected. Speciﬁcally, in the training phase, the SNN will learn real-valued

spikes and a shared convolution kernel for every output feature map. While in

the inference phase, every real-valued spike will be transformed into a binary

spike by folding a part of the value to its corresponding kernel weight. Due to

the diversity of the real-valued spikes, by absorbing part of the value from each

real spike, the original convolution kernel shared by each output map can be

converted into multiple forms. Thus diﬀerent convolution kernels for each fea-

ture map of SNNs can be obtained indirectly. It can be guaranteed theoretically

that the Real Spike method can improve the performance due to the richer

representation capability of real-valued spikes than binary spikes (see details in

Sec. 3.4). Besides, Real Spike is well compatible with present DNN-oriented

programming frameworks, and it still retains the advantages of DNN-oriented

frameworks in terms of the convolution kernel sharing mechanism in the train-

ing. Furthermore, we extract the essential idea of training-inference-decoupled

and extend Real Spike to a more generalized form, which is friendly to both

neuromorphic and non-neuromorphic hardwares (see details in Sec. 3.5). The

overall workﬂow of the proposed method is illustrated in Fig. 2.

Our main contributions are summarized as follows:

–We propose the Real Spike, a simple yet eﬀective method to obtain SNNs

with unshared convolution kernels. The Real Spike-SNN can be trained in

DNN-oriented frameworks directly. It can eﬀectively enhance the information

representation capability of the SNN without introducing training diﬃculty.

–The convolution kernel re-parameterization is introduced to decouple a

training-time SNN with real-valued spikes and shared convolution kernels,

4 Guo, Y. et al.

1.9

0.7

1.0

1.2

Real Spikes Convolution

kernel

Input Feature

Map

Output Map

1.0

Binary Spikes Convolution

kernel

Input Feature

Map

Output Map

Convolution kernel

Re-parameterization

Training Phase Inference Phase

Fig. 2. The overall workﬂow of Real Spike. In the training phase, the SNN learns

real-valued spikes and the shared convolution kernel for each output feature map. In

the inference phase, the real spikes can be converted to binary spikes via convolution

kernel re-parameterization. Then an SNN with diﬀerent convolution kernels for every

feature map is obtained.

and an inference-time SNN with binary spikes and unshared convolution

kernels.

–We extend the Real Spike to other diﬀerent granularities (layer-wise and

channel-wise). These extensions can keep shared convolution kernels in the

inference and show advantages independent of speciﬁc hardwares.

–The eﬀectiveness of Real Spike is veriﬁed on both static and neuromorphic

datasets. Extensive experimental results show that our method performs

remarkably.

2 Related Work

This work starts from training more accurate SNNs with unshared convolu-

tion kernels for each feature map. Considering the lack of specialized suitable

platforms that can support the training of deep SNNs, powerful DNN-oriented

programming frameworks are adopted. To this end, we brieﬂy overview recent

works of SNNs in three aspects: (i) learning algorithms of SNNs; (ii) SNN pro-

gramming frameworks; (iii) convolutions.

2.1 Learning Algorithms of SNNs

The learning algorithms of SNNs can be divided into three categories: convert-

ing ANN to SNN (ANN2SNN) [2,13,36,25], unsupervised learning [6,14], and

supervised learning [16,29,26,37,12]. ANN2SNN converts a pre-trained ANN to

an SNN by transforming the real-valued output of the activation function to bi-

nary spikes. Due to the success of ANNs, ANN2SNN can generate an SNN in a

short time with competitive performance. However, the converted ANN inherits

the limitation of ignoring rich temporal dynamic behaviors from DNNs, it can-

not handle neuromorphic datasets well. On the other hand, ANN2SNN usually

requires hundreds of timesteps to approach the accuracy of pre-trained DNNs.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

RealSpike:LearningReal-valuedSpikesforSpikingNeuralNetworksYufeiGuo⋆,LiwenZhang⋆,YuanpeiChen,XinyiTong,XiaodeLiu,YingLeiWang,XuhuiHuang,andZheMaIntelligentScience&TechnologyAcademyofCASIC,Beijing100854,Chinayfguo@pku.edu.cn,lwzhang9161@126.com,starhxh@126.com,mazhethu@163.comAbstract.Brain-inspireds...

展开>> 收起<<

Real Spike Learning Real-valued Spikes for Spiking Neural Networks Yufei Guo Liwen Zhang Yuanpei Chen Xinyi Tong Xiaode Liu YingLei.pdf

共18页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Real Spike Learning Real-valued Spikes for Spiking Neural Networks Yufei Guo Liwen Zhang Yuanpei Chen Xinyi Tong Xiaode Liu YingLei

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: