Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge Ashwin Bhat Adou Sangbone Assoa Arijit Raychowdhury

2025-05-06 0 0 914.1KB 6 页 10玖币

侵权投诉

Gradient Backpropagation based Feature Attribution

to Enable Explainable-AI on the Edge

Ashwin Bhat, Adou Sangbone Assoa, Arijit Raychowdhury

School of Electrical and Computer Engineering

Georgia Institute of Technology, Atlanta, GA, USA

(ashwinbhat, aassoa3)@gatech.edu, arijit.raychowdhury@ece.gatech.edu

Abstract—There has been a recent surge in the ﬁeld of

Explainable AI (XAI) which tackles the problem of providing

insights into the behavior of black-box machine learning models.

Within this ﬁeld, feature attribution encompasses methods which

assign relevance scores to input features and visualize them

as a heatmap. Designing ﬂexible accelerators for multiple such

algorithms is challenging since the hardware mapping of these

algorithms has not been studied yet. In this work, we ﬁrst analyze

the dataﬂow of gradient backpropagation based feature attribu-

tion algorithms to determine the resource overhead required over

inference. The gradient computation is optimized to minimize the

memory overhead. Second, we develop a High-Level Synthesis

(HLS) based conﬁgurable FPGA design that is targeted for

edge devices and supports three feature attribution algorithms.

Tile based computation is employed to maximally use on-chip

resources while adhering to the resource constraints. Represen-

tative CNNs are trained on CIFAR-10 dataset and implemented

on multiple Xilinx FPGAs using 16-bit ﬁxed-point precision

demonstrating ﬂexibility of our library. Finally, through efﬁcient

reuse of allocated hardware resources, our design methodology

demonstrates a pathway to repurpose inference accelerators

to support feature attribution with minimal overhead, thereby

enabling real-time XAI on the edge.

Index Terms—Convolution Neural Network, Explainable

Machine Learning, Back-propagation, Hardware Accelerator,

FPGA, High-Level Synthesis (HLS)

I. INTRODUCTION

There has been an exponential surge in the ﬁeld of machine

learning (ML) and artiﬁcial intelligence (AI) in the past

decade. ML techniques, especially Deep Neural Networks

(DNN) have found widespread adoption in various domains

such as computer vision, speech recognition, autonomous

driving and bio-medical applications. However, one major

hurdle currently is the inability to interpret the output of

these models since they are treated as a ”black-box”. The

lack of transparency in the model’s decision making process

severely limits its applicability (Fig 1). In order to address

this issue, several techniques have been proposed recently to

interpret these models [1]. Explainable-AI (XAI) methods shed

light into the workings of ”black-box” models and thereby

identify failure modes, establish trust in the end user and would

eventually enable machine teaching [2].

This work was supported by Semiconductor Research Corporation (SRC)

Task 2969.001.

To appear in 30th IFIP/IEEE International Conference on Very Large Scale

Integration (VLSI-SoC 2022)

XAI techniques can be broadly classiﬁed into three cat-

egories namely (1) visualization (2) model distillation and

(3) training interpretable models [3]. Among these three,

visualization is the only post-hoc explanation method that can

be directly applied on existing pre-trained models [4], and

hence is the focus of this work. Visualization comprises of

assigning relevance scores to the input features of the model

in order to quantify their importance to the output of the

black-box model. In the case of image classiﬁcation using

Convolutional Neural Networks (CNN), the feature attribution

scores can be visualized as a heatmap of the input pixels.

This would highlight regions that contributed most for that

particular input-output mapping produced by the model.

Fig. 1: Pitfalls of using models as black-box functions v/s

advantages of developing Explainable AI.

The different feature attribution methods comprise of two

common steps. First, a forward pass (FP) through the model to

determine the inference result. The second step is a backpropa-

gation (BP) through the model to evaluate the relevance scores

for input features and generate the heatmap (Fig. 2). Compared

to neural network training, feature attribution does not require

calculating gradient with respect to the model parameters

for the weight update (WU) step. Thus, the dataﬂow of

feature attribution algorithms (FP+BP) lies in between that

of inference (FP) and training (FP+BP+WU).

While inference accelerators have been designed for edge

applications, supporting on-device training is challenging be-

cause of the large compute and memory overheads. WU is

the most expensive step in training. It requires storing all

intermediate activations during FP (memory overhead) and

calculating gradient with respect to each model parameter

arXiv:2210.10922v1 [cs.AR] 19 Oct 2022

Fig. 2: The two phases of feature attribution algorithms for

CNNs. First, a forward pass to determine inference output.

Second, a backward pass to compute activation gradients.

(compute overhead). However, feature attribution only involves

calculating local activation gradients layer by layer (BP).

XAI has been deployed for applications such as hardware

security [5], medicine [6] and ﬁnance [7]. However, challenges

remain that prohibit end-users from accessing explanations in

real time [7]. In this work, we try to answer the question

whether real-time XAI can be supported on edge devices.

Speciﬁcally, by studying the hardware mapping of gradient

backpropagation based feature attribution methods, the paper

makes the following key contributions:

•Dataﬂow analysis of three gradient backpropagation

based feature attribution methods: (1) Saliency Map, (2)

DeconvNet, and (3) Guided Backpropagation to deter-

mine their h/w resource overhead compared to inference.

•We propose a hardware design that efﬁciently reuses com-

pute blocks and on-chip buffers (designed for inference)

during the BP step for feature attribution. The design

can be conﬁgured to support any of the three feature

attribution methods.

•We prototype our proposed design on a tiny, resource-

constrained FPGA using High-Level Synthesis (HLS),

thereby enabling real-time XAI on edge devices.

II. FEATURE ATTRIBUTION

Feature attribution methods visualize the contribution of

input features to the model’s output decision in the form of

a heatmap (Fig. 3). Higher relevance scores imply that those

corresponding features create maximum response or stimula-

tion inﬂuencing the model’s output. These post-hoc methods

can be applied to any off-the-shelf DNN model. After the

inference step (FP) to evaluate the output, a backpropagation

step (BP) is applied to evaluate gradient signals and pass them

from output to input in a layer by layer fashion. We study the

dataﬂow of three different commonly used feature attribution

methods: (1) Saliency Map [8] (2) DeconvNet [9] and (3)

Guided Backpropagation [10]. These methods differ in the

handling of the gradient signals when it encounters a ReLU

activation (Fig. 4) layer in the DNN. Equation 1 describes

how the network activations are computed during FP when it

encounters a ReLU activation at layer L.

fL+1

i=ReLU(fL

i) = max(fL

i,0) (1)

Fig. 3: An illustration [11] of post-hoc feature attribution meth-

ods. The generated relevance score are visualized as heatmaps.

These heatmaps are visually validated to be highlighting those

pixels that are relevant to the model’s output decision. In this

work, we focus on gradient, deconvnet and guided backprop.

A. Saliency Map

Saliency map is a baseline gradient based approach which

assigns relevance scores (Ri(x)) to input features based on

the partial derivative of the model’s output (fc(x), where c

is the output class) with respect to each input feature (xi) as

shown in Equation 2. A large value of the gradient implies that

small changes in the value of that input feature would produce

large change in the model’s output, thereby indicating higher

sensitivity. If we consider absolute value of the gradients,

the positive and negative contributing features cannot be

differentiated.

Ri(x) = ∂fc(x)

∂xi

(2)

During BP, when a ReLU activation is encountered, the

gradient signals are zeroed out corresponding to the negative

values of activations during FP as shown in Equation 3. Thus,

we need to store the indices of the negative activation values

in order to support BP for a ReLU activation.

i= (fL

i>0) RL+1

i, where RL

i=∂fout

∂fL

(3)

B. DeconvNet

Deconvolution was originally designed to reconstruct the

input of a CNN starting from the network outputs, in an

unsupervised manner. It has been widely adopted as an XAI

technique owing to its visualization power of most discrim-

inative features. DeconvNet consists of inverse operations of

the FP through a CNN. During BP, the convolutional layers

are replaced with deconvolutions and max-pooling layers are

replaced with unpooling layers. Deconvolution can be viewed

as a transposed convolution and hence, DeconvNet boils down

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

GradientBackpropagationbasedFeatureAttributiontoEnableExplainable-AIontheEdgeAshwinBhat,AdouSangboneAssoa,ArijitRaychowdhurySchoolofElectricalandComputerEngineeringGeorgiaInstituteofTechnology,Atlanta,GA,USA(ashwinbhat,aassoa3)@gatech.edu,arijit.raychowdhury@ece.gatech.eduAbstractTherehasbeenarecen...

展开>> 收起<<

Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge Ashwin Bhat Adou Sangbone Assoa Arijit Raychowdhury.pdf

共6页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge Ashwin Bhat Adou Sangbone Assoa Arijit Raychowdhury

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: