Black Box Model Explanations and the Expectation of Human Interpretation - An Analyzes in the Context of Homicide Prediction

2025-05-06 0 0 2.44MB 44 页 10玖币

侵权投诉

Black Box Model Explanations and the

Expectation of Human Interpretation - An

Analyzes in the Context of Homicide Prediction

Jos´e de Sousa Ribeiro Filho1,2,3∗[0000−0002−8836−4188], Nikolas Jorge Santiago

Carneiro3[0000−0002−5097−0772], Lucas Felipe Ferraro

Cardoso2,3[0000−0003−3838−3214], and Ronnie Cley de Oliveira

Alves1,3[0000−0003−4139−0562]

1Federal University of Par´a (UFPA), Bel´em, Brazil

2Federal Institute of Education, Science and Technology of Par´a (IFPA),

Ananindeua, Brazil

3Vale Institute of Technology (ITV DS), Bel´em, Brazil

∗Corresponding Author:

E-mail: jose.ribeiro@ifpa.edu.br

Phone: +55 91 98185-3166

Abstract. Strategies based on Explainable Artiﬁcial Intelligence (XAI)

have promoted better human interpretability of the results of black box

models. This opens up the possibility of questioning whether explana-

tions created by XAI methods meet human expectations. The XAI meth-

ods being currently used (Ciu,Dalex,Eli5,Lofo,Shap, and Skater ) pro-

vide various forms of explanations, including global rankings of relevance

of features, which allow for an overview of how the model is explained as

a result of its inputs and outputs. These methods provide for an increase

in the explainability of the model and a greater interpretability grounded

on the context of the problem. Intending to shed light on the explanations

generated by XAI methods and their interpretations, this research ad-

dresses a real-world classiﬁcation problem related to homicide prediction,

already peer-validated, replicated its proposed black box model and used

6 diﬀerent XAI methods to generate explanations and 6 diﬀerent human

experts. The results were generated through calculations of correlations,

comparative analysis and identiﬁcation of relationships between all ranks

of features produced. It was found that even though it is a model that is

diﬃcult to explain, 75% of the expectations of human experts were met,

with approximately 48% agreement between results from XAI methods

and human experts. The results allow for answering questions such as:

“Are the Expectation of Interpretation generated among diﬀerent human

experts similar? ”, “Do the diﬀerent XAI methods generate similar ex-

planations for the proposed problem? ”, “Can explanations generated by

XAI methods meet human expectation of Interpretations? ”, and “Can

Explanations and Expectations of Interpretation work together?”.

Keywords: Explainable Artiﬁcial Intelligence ·Black Box Model ·Hu-

man in the Loop ·Homicide prediction ·Machine Learning

arXiv:2210.10849v2 [cs.LG] 4 Jul 2024

2 J. Ribeiro et al.

1 Introduction

In recent years, technology has increasingly evolved and allowed intelligent algo-

rithms to be present in our daily lives through solutions to the most diverse types

of problems, thus further requiring that machine learning models solve increas-

ingly complex problems provinding conﬁdent explainabilities of their decisions

[81,34].

Computational models based on bagging and boosting algorithms, because

they provide high performance and high generalization capacity, are commonly

used in computing to solve regression and classiﬁcation problems based on tab-

ular data. However, these models are not considered transparent algorithms4,

being considered black box algorithms5and, therefore, are less used in problems

related to sensitive contexts, such as health and safety [91,55].

By observing the most recent literature on Explainable Artiﬁcial Intelligence

(XAI) [40], the use of black box algorithms in sensitive real-world contexts re-

quires conﬁdence (on the part of the human user) to be gained in the predictions

of this type of algorithm. In this sense, diﬀerent strategies have been developed

on two knowledge fronts: one aimed at generating greater explanations of the

model itself; and other front with analyzes concerning the interpretation of the

explanations produced (interpretations made by a human user) [15,65,36].

Black box model explanations are created through analyzes Model Agnostic6

or Model Speciﬁc7, also referred to as Model Inductions [66,39] or even Post-hoc

Analyzes [15], since in this type of technique only the training data, test data,

the model itself and its outputs are used for creating explanations.

The limited understanding of black box models requires the search for meth-

ods and tools that can provide information about local explanations — aiming

at predicting around an instance through various methods to obtain a local fea-

ture relevance ranking [62] — and global explanations — when it is possible to

understand the rationale of all instances of the model by generating a global

feature relevance ranking [62,38] — as a means of making interpretable, and

thus more reliable, decisions [39].

The term Ethical AI [67], has been growing in the area of machine learning

in recent years, which shows the concern of the computing community with

the development and use of models that are based on responsible and reliable

practices in the use of AI. As a result, guidelines, tools and new methods have

emerged with the aim of explaining machine learning models, making them more

reliable, since a human can only trust what they can understand [9].

The terminologies Feature Relevance Ranking and Feature Importance Rank-

ing are widely used as synonyms in the computing community, but have diﬀerent

4Transparent Algorithms: Algorithms that generate explanations for how a given

output was produced. Examples: Decision Tree, Logistic Regression and K-nearest

Neighbors [15].

5Black Box Algorithms: Machine learning algorithms that have classiﬁcation or re-

gression decisions that are hidden from the user [26].

6Model Agnostic: does not depend on the type of machine learning model [66].

7Model Speciﬁc: depend of the one speciﬁc type of machine learning model [52].

Title Suppressed Due to Excessive Length 3

deﬁnitions in XAI study area, as shown in [15]. Since feature rankings are re-

garded as ordered structures whereby each feature of the dataset used by the

model appears in a position indicated by a score. The main diﬀerence being that,

in relevance ranking, the calculation of the score is based on the model output,

whereas to calculate the importance ranking of features, the correct label to be

predicted is used [15,66].

In previous studies [76], evidence was veriﬁed that shows the existence of

models (datasets and algorithms) that are easy to explain and also diﬃcult,

through analyzes involving 82 diﬀerent models (diﬀerent algorithms and datasets).

Since the use of several XAI methods in explaining a single model can allow the

generation of diﬀerent explanations based on relevance ranks — which show

that the model is diﬃcult to explain — or even similar explanations between the

methods — which show that the models is easy to explain.

Seeking to continue the results found in [76], this article carries out speciﬁc

studies from the perspective of just one model, duly evaluated in [75], seeking

to bring information about the context in which the model is inserted and how

these aspects imply in their explanations aiming for greater conﬁdence in the

model.

A technique is also presented, called ConeXi, which allows the combination of

diﬀerent explanations coming from XAI methods or even human people (called

here Expectation of Interpretation). Enabling the insertion of humans in the

explanation process, as in [86,69,84,92].

Given this context and the various research fronts involving explanations,

interpretations and human interactions in the black box opening process8, the

following questions arise:

–“Do the diﬀerent XAI methods generate similar explanations for the proposed

problem?”;

–“Are the expectation of interpretation generated among diﬀerent human ex-

perts similar?”;

–“Can explanations generated by XAI methods meet human expectation of

interpretations?”;

–“Can Explanations and Expectations of Interpretation work together?”.

By seeking to answer these questions, an experiment was developed that uses

the machine learning model of homicide prediction advocated in [75], and from

this, the 6 rankings explanations were generated by means of XAI methods, as

in [76], and 6 ranks expectation of interpretations generated by diﬀerent human

experts.

Then, comparisons and identiﬁcation of existing relationships between all

pairs of ranks created were performed to ﬁnd the desired answers. Finally, the

generated ranks were combined into a single overall rank by means of a technique

proposed hereby based on the results of the explanations of the XAI methods

and expectation of interpretations.

8Black box opening: Set of methods, strategies and processes used to make black box

models explainable [26]

4 J. Ribeiro et al.

The main contributions of the research to the area of machine learning, which

can be completely replicated or used for other research and contexts, are:

–Discussion regarding the similarity of explanations generated by XAI meth-

ods and their interpretability, focusing on the speciﬁc context-sensitive prob-

lem — homicides prediction — in order to measure whether the XAI methods

explain the model as expected by human experts;

–Concept of Expectation of Interpretation, which in general terms is the in-

terpretation expected by an expert of a real-world problem based on their

knowledge of the problem and the working principle of the machine learning

model being analyzed;

–The ConeXi, a tool to combine Expectation of Interpretation with explana-

tions by XAI methods, which are based on global feature relevance rankings,

in order to build a Collaborative Explanation of the model using human ex-

pert knowledge and diﬀerent XAI methods, i.e. human and machine.

–Overall methodology developed by this study as a deliverable, as it promotes

data used, code developed, results collected, and the repositories created, in

accordance with the Fair Guiding Principles for scientiﬁc data management

and stewardship.

2 Background

This section will present: The concepts of explainability and interpretability in

XAI; The operating principles of the XAI methods based in relevance ranks; And

aspects referring to previous research on models considered easy and diﬃcult to

explain.

2.1 Explainability and Interpretability in XAI

The concepts of explainability and interpretability in machine learning are con-

siderably close and even complement one another [15,66]. Therefore, it is of

utmost importance that they are presented and diﬀerentiated.

Explanability is associated with the explanatory interface between a com-

putational model and a human, which aids in the decision-making process as it

seeks to make the model understandable [15,66].

Interpretability is the ability to provide meaning in terms that are under-

standable to a human being, or even the attempt to interpret an explanation

[15,83,65,66,36].

Based on these two concepts, which are widespread in the area of machine

learning, it is understood that in a practical way explainability seeks to create

subsidies that explain the black box model9in a technical manner, whereas

interpretability is used-centric wich meaning to the explanations created for a

human user, such meaning being based on the context of the problem and the

knowledge of the individual [66,15].

9Explain the black box model: Also known as the process of “opening the black box”.

Title Suppressed Due to Excessive Length 5

Both explainability and interpretability of models are fundamental pieces

in the decision-making process, as they provide the end user with support in

detecting various problems or even biases in the data being used by the model

[15,83].

It is not possible to conduct a study involving analyzes of explainability

and interpretability of computer models without considering the speciﬁc con-

text/problem in which they are embedded and the human factors as well [15].

In this sense, this research focuses on a single speciﬁc problem to perform its

analyzes. In addition to this and also to issues of time and cost feasibility, the

context of homicide prediction was chosen.

Therefore, it can be assumed that explainability and interpretability allow

the generation of reliability, understanding, and fairness to black box machine

learning models. In the studies and experiments described herein, the main fo-

cus is on the explainability of each generated model and its relationship to the

interpretabilities (in this case, expectations) generated by humans in the context

of crime prediction.

2.2 Methods of Explainable Artiﬁcial Intelligence

In recent years, there has been an increasing need to explain black box ma-

chine learning models in an agnostic and speciﬁc manner. Among the various

initiatives present in the literature, there is a greater number of XAI methods

developed speciﬁcally for neural networks, whereas a smaller number of methods

are speciﬁcally developed for tree-ensemble algorithms [73,53,58,2].

The need to obtain greater conﬁdence in black box models, currently the

community in the XAI area has been developing various methods, concepts,

techniques and tools in order to carry out the process of explaining these models.

Thus, it is argued that from the creation of layers of explanations on the model,

a human user can create their interpretations and thus better understand how

the decisions taken by the model were carried out, obtaining greater conﬁdence

at the end of the process [15,66].

The so-called post-hoc explanation is the currently most widely used existing

XAI method category in the computing community. Their main peculiarity is

the fact that they only use training data, test data, model output data and the

model itself, already duly trained, to generate the explanations[15].

According to [66], the post-hoc XAI techniques can be divided into dif-

ferent strategies: Text Explanations,Visual Explanations,Local Explanations,

Explanation-by-simpliﬁcation,Feature Relevance Explanations and Explanation-

by-example. Based on these types of methods, this research focus only on the

Feature Relevance Explanations, because the ranking structure makes it possible

to carry out a quantitative comparative analysis of the explanations generated.

Based on the above, this research conducted a bibliographic and practical

survey (development) on the main existing XAI methods, speciﬁcally aimed

at generating model-agnostic or model-speciﬁc global explanation ranks that

support tabular data and tree-ensemble algorithm.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

BlackBoxModelExplanationsandtheExpectationofHumanInterpretation-AnAnalyzesintheContextofHomicidePredictionJos´edeSousaRibeiroFilho1,2,3∗[0000−0002−8836−4188],NikolasJorgeSantiagoCarneiro3[0000−0002−5097−0772],LucasFelipeFerraroCardoso2,3[0000−0003−3838−3214],andRonnieCleydeOliveiraAlves1,3[0000−0003...

展开>> 收起<<

Black Box Model Explanations and the Expectation of Human Interpretation - An Analyzes in the Context of Homicide Prediction.pdf

共44页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Black Box Model Explanations and the Expectation of Human Interpretation - An Analyzes in the Context of Homicide Prediction

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: