A N OVEL APPROACH FOR NEUROMORPHIC VISION DATA COMPRESSION BASED ON DEEPBELIEF NETWORK Sally Khaidem

2025-04-30 0 0 2.79MB 7 页 10玖币

侵权投诉

A NOVEL APPROACH FOR NEUROMORPHIC VISION DATA

COMPRESSION BASED ON DEEP BELIEF NETWORK

Sally Khaidem

Department of Electrical Engineering

Indian Institute of Technology, Madras

Chennai, 600036, India

ee20d041@smail.iitm.ac.in

Mansi Sharma

Department of Electrical Engineering

Indian Institute of Technology, Madras

Chennai, 600036, India

mansisharma@ee.iitm.ac.in

Abhipraay Nevatia

Department of Mechanical Engineering

Indian Institute of Technology, Madras

Chennai, 600036, India

me20b007@smail.iitm.ac.in

October 28, 2022

ABSTRACT

A neuromorphic camera is an image sensor that emulates the human eyes capturing only changes in

local brightness levels. They are widely known as event cameras, silicon retinas or dynamic vision

sensors (DVS). DVS records asynchronous per-pixel brightness changes, resulting in a stream of

events that encode the brightness change’s time, location, and polarity. DVS consumes little power

and can capture a wider dynamic range with no motion blur and higher temporal resolution than

conventional frame-based cameras. Although this method of event capture results in a lower bit rate

than traditional video capture, it is further compressible. This paper proposes a novel deep learning-

based compression scheme for event data. Using a deep belief network (DBN), the high dimensional

event data is reduced into a latent representation and later encoded using an entropy-based coding

technique. The proposed scheme is among the ﬁrst to incorporate deep learning for event compression.

It achieves a high compression ratio while maintaining good reconstruction quality outperforming

state-of-the-art event data coders and other lossless benchmark techniques.

Keywords Event computing ·entropy coding ·dynamic vision sensor ·deep belief network

1 Introduction

Sight, along with the brain, is the dominant sense in humans for perceiving the world and learning new things. “Silicon

Retina” [

] mimics the neural architecture of human eyes and reveals a new, powerful way of computations, sparking

the emerging ﬁeld of neuromorphic engineering. Bio-inspired novel sensors such as Dynamic Vision Sensors (DVS) [

]

measure intensity changes asynchronously rather than capturing intensity images at a ﬁxed rate. As a result, it generates

a stream of events that encodes the time, location, and polarity of brightness changes, where the data rate depends

on scene complexity and camera speed. When compared to traditional cameras, DVS have superior properties. They

have a very high dynamic range (140 dB versus 60 dB), no motion blur, and measurements with latency on the order

of microseconds. DVS devices, such as Dynamic and Active-pixel Vision Sensor (DAVIS) [

] and Asynchronous

Time-based Image Sensor (ATIS) [

] are a viable alternative in challenging conditions for standard cameras, such as

high-speed high-dynamic-range motion photography, robotic automation, and intelligent surveillance [5, 6, 7, 8].

The neuromorphic silicon technology uses Address-Event-Representation (AER) [

], a communication protocol for

transferring spikes events between bio-inspired chips. A tuple

(X, Y, p, t)

represents each event, where

and

denote

arXiv:2210.15362v1 [cs.CV] 27 Oct 2022

APREPRINT - OCTOBER 28, 2022

(a) (b)

Figure 1: Visualization of Box event output in space-time. Red and Blue represents events with polarity ‘

’ and ‘

’

respectively. (a) without polarity separation (b) with polarity separation and creation of super-frame.

the location of the event at a particular timestamp

with polarity

indicating an increase or decrease in event brightness.

Each tuple is represented by 64 bits, with the timestamp being 32 bits and the remaining three ﬁelds being 32 bits. The

goal is to gather helpful information from event data and utilize it for processing.

DVS acquire information asynchronously and sparsely, with high temporal resolution and low latency. Hence, the

temporal aspect, particularly latency, is critical in the event data processing. The output stream cannot use traditional

vision algorithms since it is a series of asynchronous events rather than actual intensity images. Therefore, development

of new algorithms that take advantage of the sensor’s high temporal resolution and asynchronous nature is necessary.

There are two types of algorithms based on the processing number of events at the same time. The ﬁrst approach

operates on an event-by-event basis, in which the system’s state changes upon the occurrence of a single event, resulting

in minimal latency. The second approach involves latency because it operates on groups or packets of events. It can still

provide a system state update upon the occurrence of each event if the window moves by one event at a time. The data

storage and transmission bandwidth limitation for onboard DVS processing is an open challenge and requires immediate

solutions. Spike coding [

] is a dedicated lossless compression strategy that exploits event data’s time-series and

asynchronous nature. It follows a cube-based coding framework where the spike sequence is divided into multiple

macro-cubes and encoded accordingly. Entropy-based coding strategies like Huffman and Arithmetic can effectively

encode DVS data by treating each spike event ﬁeld as an input symbol. Existing lossless coding schemes such as

dictionary-based [

] and fast-integer [

] encoders can also compress the DVS data after converting the

spike events into a multivariate stream of integers.

The applications of DVS range from self-driving cars [

] to robotics [

] and drones [

]. Applications such as

coordinating multiple intelligent vehicles (IoV) (cars, drones, etc.) having onboard processing constraints require

real-time data sharing and feedback. In comparison to traditional sensing techniques, neuromorphic sensing provides an

intrinsic compression. Further compression of event data is advantageous for transmission in the Internet of Things

(IoT) and the Internet of IoV. This paper presents a novel approach suitable for DVS data compression based on a

deep learning algorithm, Deep Belief Network (DBN). Figure 2 depicts the complete workﬂow of event compression.

The entire stream of events is converted into a dimensionally reduced latent representation by multiple code layer

blocks using the DBN. The compact latent code blocks contain recurring information suitable for lossless symbol-based

encoders. Hence, we compress the latent code using an entropy-based Huffman coding technique. The primary

contributions of the proposed scheme are as follows:

•

The proposed framework is among the ﬁrst to incorporate deep learning techniques for event data processing.

High-dimensional event data is transformed into low-dimensional latent code using a multilayer neural

network called a deep belief network. We perform lossless encoding of low-dimensional latent features using

entropy-based encoders to achieve a further compressed representation.

•

We formulated a unique events arrangement deemed more suitable for processing by the proposed framework.

The events are time-aggregated by accumulating spike events over time as super-frame sequences, as explained

in Section 2.1. Super-frames result in high spatial and temporal correlation among the event data.

•

We conducted extensive comparisons with lossless benchmark strategies on a diverse standard dataset with

varying scene complexity and camera movement. As a result of the learning-based framework, we obtain a

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ANOVELAPPROACHFORNEUROMORPHICVISIONDATACOMPRESSIONBASEDONDEEPBELIEFNETWORKSallyKhaidemDepartmentofElectricalEngineeringIndianInstituteofTechnology,MadrasChennai,600036,Indiaee20d041@smail.iitm.ac.inMansiSharmaDepartmentofElectricalEngineeringIndianInstituteofTechnology,MadrasChennai,600036,Indiamans...

展开>> 收起<<

A N OVEL APPROACH FOR NEUROMORPHIC VISION DATA COMPRESSION BASED ON DEEPBELIEF NETWORK Sally Khaidem.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

A N OVEL APPROACH FOR NEUROMORPHIC VISION DATA COMPRESSION BASED ON DEEPBELIEF NETWORK Sally Khaidem

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: