
Inspired by biological systems, research on event-driven perception has started to gain momentum,
and several asynchronous event-based sensors have been proposed, including event cameras Gallego
et al. (2020) and event-based tactile sensors Taunyazoz et al. (2020). In contrast to standard synchronous
sensors, such event-based sensors can achieve higher energy efficiency, better scalability, and lower
latency. However, due to the high sparsity and complexity of event-driven data, learning with these
sensors is still in its infancy Pfeiffer and Pfeil (2018). Recently, several works Taunyazoz et al. (2020);
Gu et al. (2020); Taunyazov et al. (2020) utilized Spiking Neural Networks (SNNs) Shrestha and Orchard
(2018); Pfeiffer and Pfeil (2018); Xu et al. (2021) to tackle event-driven tactile learning. Unlike ANNs,
which require expensive transformations from asynchronous discrete events to synchronous real-valued
frames, SNNs can process event-based sensor data directly. Moreover, unlike ANNs that employ artificial
neurons Maas et al. (2013); Xu et al. (2015); Clevert et al. (2015) and conduct real-valued computations,
SNNs adopt spiking neurons Gerstner (1995); Abbott (1999); Gerstner and Kistler (2002) and utilize
binary 0-1 spikes to process information. This difference reduces the mathematical dot-product operations
in ANNs to less computationally expensive summation operations in SNNs Roy et al. (2019). Due to
the advantages of SNNs, these works are always energy-efficient and suitable for power-constrained
devices. However, due to the limited representative abilities of existing spiking neuron models and high
spatio-temporal complexity in the event-based tactile data Taunyazoz et al. (2020), these works still cannot
sufficiently capture spatio-temporal dependencies and thus hinder the performance of event-driven tactile
learning.
In this paper, to address the problems mentioned above, we make several contributions that boost event-
driven tactile learning, including event-driven tactile object recognition and event-driven slip detection.
We summarize a list of acronyms and notations in Table 7. Please refer to it during the reading.
First, to enable richer representative abilities of existing spiking neurons, we propose a novel
neuron model called “location spiking neuron”.
Unlike existing spiking neuron models that update
their membrane potentials based on time steps Roy et al. (2019), location spiking neurons update
their membrane potentials based on locations. Specifically, based on the Time Spike Response Model
(TSRM) Gerstner (1995), we develop the “Location Spike Response Model (LSRM)”. Moreover, to make
the location spiking neurons more applicable to a wide range of applications, we develop the “Location
Leaky Integrate-and-Fire (LLIF)” model based on the most commonly-used Time Leaky Integrate-and-
Fire (TLIF) model Abbott (1999). Please note that TSRM and TLIF are the classical Spike Response
Model (SRM) and Leaky Integrate-and-Fire (LIF) in the literature. We add the character “T (Time)” to
highlight their differences from LSRM and LLIF. These location spiking neurons enable the extraction of
feature representations of event-based data in a novel way. Previously, SNNs adopted temporal recurrent
neuronal dynamics to extract features from the event-based data. With location spiking neurons, we can
build SNNs that employ spatial recurrent neuronal dynamics to extract features from the event-based data.
We believe location spiking neuron models can have a broad impact on the SNN community and spur the
research on spike-based learning from event sensors like NeuTouch Taunyazoz et al. (2020), Dynamic
Audio Sensors Anumula et al. (2018), or Dynamic Vision Sensors Gallego et al. (2020).
Next, we investigate the representation effectiveness of location spiking neurons and propose
two models for event-driven tactile learning.
Specifically, to capture the complex spatio-temporal
dependencies in the event-driven tactile data, the first model combines a fully-connected (FC) SNN
with TSRM neurons and a fully-connected (FC) SNN with LSRM neurons, henceforth referred to as the
Hybrid SRM FC
. To capture more spatio-temporal topology knowledge in the event-driven tactile data,
the second model fuses the spatial spiking graph neural network (GNN) with TLIF neurons and temporal
spiking graph neural network (GNN) with LLIF neurons, henceforth referred to as the
Hybrid LIF GNN
.
To be more specific, the Hybrid LIF GNN first constructs tactile spatial graphs and tactile temporal
graphs based on taxel locations and event time sequences, respectively. Then, it utilizes the spatial spiking
graph neural network with TLIF neurons and the temporal spiking graph neural network with LLIF
neurons to extract features of these graphs. Finally, it fuses the spiking tactile features from the two
networks and provides the final tactile learning prediction. Besides the novel model construction, we
also specify the location orders to enable the spatial recurrent neuronal dynamics of location spiking
neurons in event-driven tactile learning. In addition, we explore the robustness of location orders on event-
driven tactile learning. Moreover, we design new loss functions involved with locations and utilize the
backpropagation methods to optimize the proposed models. Furthermore, we develop the timestep-wise
inference algorithms for the two models to show their applicability to the spike-based temporal data.
2/24