1 Local Interpretable Model Agnostic Shap Explanations for machine learning models

2025-04-28 0 0 1.42MB 21 页 10玖币

侵权投诉

Local Interpretable Model Agnostic Shap Explanations for machine learning

models

P. Sai Ram Aditya1,2, Mayukha Pal1,$

1ABB Ability Innovation Center, Asea Brown Boveri Company, Hyderabad 500084, India.

2Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, Kandi,

Sangareddy, Telangana 502285, India.

$Corresponding author:

Dr. Mayukha Pal

R&D Principal Program Manager

ABB Ability Innovation Center

10th Floor, Western Aqua, Kondapur

Hyderabad – 500084, TS, India.

Tele: +91-9866161632

Email: Mayukha.pal@in.abb.com

Abstract

With the advancement of technology for artificial intelligence (AI) based solutions and analytics

compute engines, machine learning (ML) models are getting more complex day by day. Most of

these models are generally used as a black box without user interpretability. Such complex ML

models make it more difficult for people to understand or trust their predictions. There are variety

of frameworks using explainable AI (XAI) methods to demonstrate explainability and

interpretability of ML models to make their predictions more trustworthy. In this manuscript, we

propose a methodology that we define as Local Interpretable Model Agnostic Shap Explanations

(LIMASE). This proposed ML explanation technique uses Shapley values under the LIME

paradigm to achieve the following (a) explain prediction of any model by using a locally faithful

and interpretable decision tree model on which the Tree Explainer is used to calculate the shapley

values and give visually interpretable explanations. (b) provide visually interpretable global

explanations by plotting local explanations of several data points. (c) demonstrate solution for the

submodular optimization problem. (d) also bring insight into regional interpretation e) faster

computation compared to use of kernel explainer.

Keywords: LIME, SHAP, Base value, Tree Explainer, Kernel Explainer, Explainable AI

Significance

Shapley value is a concept from game theory which has many desirable properties useful in XAI

methods but its computation for most of the complex ML models is slow with accuracy issues.

Tree explainer algorithm introduced by Lundberg et. al. computes exact shapley values efficiently

and accurately on tree-based models. Our proposed method leverages this algorithm using a

decision tree and calculate shapley values to interpret any model hence observed to be much faster

as it is locally faithful following the LIME paradigm. The method also helps in global

interpretation following the SP-LIME paradigm and additionally gives insight into regional

interpretation.

1. Introduction

Nowadays, we find in almost all domains Machine Learning is used. Due to the increase in

computation power, cloud technologies and its availability to more people, domains every day,

ML methods are being used extensively not only by individuals or industries but also in significant

domains like defense, healthcare, finance etc [1-2]. These domain users have little freedom for

error from ML models hence it is worth a discussion how much of a black box ML model could

be used for such decision making [3]. In view of this growing discussion for the ML models, XAI

techniques tries to bring many solutions to these questions [4-5]. Machine learning research in the

area of XAI is evolving rapidly [6-8].

There are many XAI methods evolved in past and continual research discussion in this area

recently for better model development to make it more explainable, interpretable and universal [9-

11]. Considerable research is being done to evaluate various XAI methods [12-14]. Local

interpretable model agnostic explanations (LIME) is a popular XAI method proposed by Marco et

al. [15]. It trains an easily interpretable model g to be locally faithful to the predictions of a complex

model f where g is further interpreted for local explanations. As g has the knowledge of the decision

boundary around the point it was trained in an example of a classification problem thus interpreting

g is same as interpreting f locally. g was taken as a simple logistic regressor in this work with its

weights giving the explanations. This method is simple yet very effective.

Shapley values is another XAI method with its roots from the game theory. It was introduced by

Lloyd Shapley in 1951 for which he later got a Nobel prize in year 2012. Shapley value is known

to have many desirable properties that many other methods lack hence it’s the most used XAI

method [16-18] but calculating them has been a problem in the field of AI. There were no methods

to calculate shapley values on complex models accurately and efficiently for ML Models until

2020 when Lundberg et al. proposed the Tree Explainer algorithm [19] that computes shapley

value exactly on any Tree based model efficiently.

In this paper, we propose a methodology to make use of LIME approach while leveraging the

goodness of shapley value simultaneously. A simple decision tree is trained to be locally faithful

for any complex model f. The tree explainer algorithm is used to give shapley explanation values

for any local point. With use of shapley and LIME, our method is also model-agnostic hence it

works without any knowledge about the model. Model-agnostic methods generally need a black

box machine learning model which could predict from a desired input. This is not only convenient

for the user but also helps in protecting the confidentiality of those complex model f when needed.

When an important decision to be made in domains like health, finance or defense, individual

interpretation is important to trust the decision of models. Our proposed method compliments such

scenario by calculating the contributions of different features that makes the decision while

demonstrating explainability. The proposed method helps in visualizing [20] these contributions

and hence could help establish trust of end user for the ML model prediction, decision.

Many a times individual interpretation is not enough when a model needs to be field deployed. It

is important that we have a global understanding of the model. The proposed method also helps in

visualizing the global explanations by plotting local explanations of several points. Local

interpretation of all the samples gives a global inference of the model but it may not be efficient

repetitively in all scenarios. We also demonstrate our method usability in implementing

Submodular Pick (SP). SP algorithm helps in selecting samples as diverse as possible based on the

budget and thus efficiently perform global interpretation using those samples. The manuscript is

organized with section 2 describing details of the materials used in our proposed methodology

while section 3 details the results and discussions using LIMASE. The section 4 of the manuscript

discusses about our inference and future scope.

2. Methods and Materials:

Let f be the complex machine learning model that needs to be interpreted and g be the simple

interpretable model we chose, to mimic f locally following the LIME paradigm. In our proposed

methodology g is a decision tree that we used.

2.1 Choice of using Shapley Values

Shapley value concept from game theory has been very reliable, desirable in XAI methods ever

since it was recently proposed [16, 21-22]. Some of its desirable properties are listed below:

Here V denotes the value function which when given a feature returns its importance. S is the set

of features while N is the total number of features.   denotes the Shapley value of i th feature

when value function V is used. In our case V is the ML predictor f. We compute shapley values

as:

  



      (1)

1) Efficiency: The sum of the contribution i.e the shapley value of each feature must add up

to the model’s prediction subtracting the average prediction of the model i.e the base value

in SHAP.

Axiom 1:    

 (2)

2) Consistency: If a feature A is consistently contributing more to the prediction than feature

B, then shapley values faithfully reflects it hence the shapley value of A will be higher than

that of B.

Axiom 2:        

    (3)

3) Missingness: A feature that does not contribute to the prediction must be attributed a zero

value.

   

     (4)

4) Symmetry: Symmetric features contribute equivalently to the final prediction and hence

have the same contribution values.

All these desirable properties of Shapley values makes it one of the most important methods

suitable for explainable AI. Calculating exact shapley values has been difficult. Many methods

were proposed to calculate approximate shapley values. Recently, it was demonstrated Tree

Explainer could calculate exact shapley values efficiently. To leverage these awesome features,

our architecture uses shapley in LIME paradigm with decision tree as the underlying tree explainer

model.

2.2 Choice of using Decision Tree

In LIME, g was either a logistic regressor in case of classification or a linear regressor in case of

regression which were interpretable by themselves. In our proposed method, we chose decision

tree as g to use the tree explainer. Decision trees are known to perform well both in regression and

classification tasks. They are robust and less sensitive to outliers. They are not only capable of

giving a well fitted model but also works well with the tree explainer.

In LIME the Fidelity-Interpretability trade-off was discussed where fidelity is the measure of how

faithful g is to f locally and interpretability is a measure of how easy it is to interpret g. In our

proposed method, the interpretability of g comes from the fact that it is a decision tree, and we use

tree explainer to calculate the exact shapley values efficiently. Fidelity comes from how well g fits

f locally. Decision trees are known to work well with less data which was observed in our

implementation as well. Decision trees mimic the actual network very well locally. The trade-off

is discussed more in section 2.3.1.

2.3 Local Interpretation

Algorithm 1 Shapley Explanation Value computation for Proposed Method

Inputs: Model f, Number of samples N, sample x, weight definition w.

P  {}

      

pi  sample_around(x)

P  P{pi, f(pi), w(pi)}

End for

Train a decision tree g on P.

Shapley Values  Tree Explainer(g, x)

Return Shapley Values

Algorithm 1 Explanation

Let x be the point where we wish to interpret f. N is the number of perturbations around x that we

use to train our decision tree g. Then we sample N points around x with x’ be one of them and

obtain f(x’) for all these points to build a dataset     . Here w is the weight

function that calculate weights of the samples in D based on their distance from x. Now train g on

D. Calculate shapley values on g at x using SHAP’s Tree Explainer. These values give an inference

about how different features contribute to predict f(x) for x. SHAP Force plot helps towards visual

interpretation of these values. The above method is repeated for multiple x’s and plotted a SHAP

summary plot for all the values by providing a global interpretation on model f. The steps are:

1. Select the sample x where we wish to explore local interpretation.

2. Sample N points around x using any distribution and get Zx.

3. Construct array of weights

4. Then, calculate f(z) for every z in Zx and get Zy.

5. Now train a Decision Tree (g) with Z = (Zx , Zy) as train data and W as sample weights.

6. Use Tree Explainer on g to calculate shapley values at sample x.

7. Draw force plot using shapley values and interpret the contributions of various features for

predicting f(x).

8. Repeat above steps for multiple points and draw the summary plot for all the values for the

global interpretation.

2.3.1 Kernel Width(σ) and Local Fidelity-Interpretability tradeoff:

A popular choice of w is to reduce the weights exponentially with 1 at x and 0 at infinite distance

from x which we practice in our experimentation too. w could be expressed as exp(−d(x, x’)2 /σ2)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1LocalInterpretableModelAgnosticShapExplanationsformachinelearningmodelsP.SaiRamAditya1,2,MayukhaPal1,$1ABBAbilityInnovationCenter,AseaBrownBoveriCompany,Hyderabad500084,India.2DepartmentofArtificialIntelligence,IndianInstituteofTechnologyHyderabad,Kandi,Sangareddy,Telangana502285,India.$Correspondi...

展开>> 收起<<

1 Local Interpretable Model Agnostic Shap Explanations for machine learning models.pdf

共21页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 Local Interpretable Model Agnostic Shap Explanations for machine learning models

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: