CHOOSING THEBESTINCENTIVES FOR BELIEF ELICITATION WITH AN APPLICATION TO POLITICAL PROTESTS Nathan Canen and Anujit Chakraborty

2025-05-01 0 0 433.51KB 24 页 10玖币

侵权投诉

CHOOSING THE BEST INCENTIVES FOR BELIEF ELICITATION

WITH AN APPLICATION TO POLITICAL PROTESTS

Nathan Canen and Anujit Chakraborty

ABSTRACT. Many experiments elicit subjects’ prior and posterior beliefs about a random

variable to assess how information affects one’s own actions. However, beliefs are multi-

dimensional objects, and experimenters often only elicit a single response from subjects. In

this paper, we discuss how the incentives offered by experimenters map subjects’ true belief

distributions to what proﬁt-maximizing subjects respond in the elicitation task. In particular,

we show how slightly different incentives may induce subjects to report the mean, mode, or

median of their belief distribution. If beliefs are not symmetric and unimodal, then using

an elicitation scheme that is mismatched with the research question may affect both the

magnitude and the sign of identiﬁed effects, or may even make identiﬁcation impossible.

As an example, we revisit Cantoni et al. (2019)’s study of whether political protests are

strategic complements or substitutes. We show that they elicit modal beliefs, while modal

and mean beliefs may be updated in opposite directions following their experiment. Hence,

the sign of their effects may change, allowing an alternative interpretation of their results.

JEL: C81, C93, D74, P00

Keywords: Belief Elicitation, Experimental Designs, Identiﬁcation, Political Protests

1. Introduction

Economic theory posits that an individual’s actions in a strategic environment depend on

their preferences over outcomes and their beliefs about how others would act. However,

measuring the causal role of beliefs on actions from observational data is inherently difﬁ-

cult. After all, even when actions are observable, beliefs are rarely observed. Meanwhile,

imputing beliefs from actions requires presupposing their causal effect, even though that

is often the empirical goal itself.

Date: October 25, 2022.

We thank Nina Bobkova, Juan Felipe Ria˜

no, Sebastian Saiegh, Ko Sugiura and Francesco Trebbi for very

helpful comments and suggestions. All errors are our own.

Canen: (Corresponding Author) University of Houston, TX, USA and Research Economist at NBER. email:

ncanen@uh.edu

Chakraborty: University of California - Davis, CA, USA. email: chakraborty@ucdavis.edu.

arXiv:2210.12549v1 [econ.EM] 22 Oct 2022

Experimental methods provide an attractive solution in these contexts. In experiments,

researchers can observe actions and elicit beliefs, both before and after carefully designed

information interventions. For example, within the context of political activism, Cantoni

et al. (2019); Jarke-Neuert et al. (2021); Hager et al. (2022a,b) all elicit subjects’ beliefs

about others’ planned participation in political events/protests. They then measure how

changing subjects’ information about others’ participation causally affects individual beliefs

and, thence, individuals’ own attendance.1

However, eliciting beliefs is fundamentally different than eliciting simpler variables, such

as willingness-to-pay. This is because beliefs are a probability distribution, often deﬁned

over a large set of outcomes, rather than just a point response (i.e., a real number between

0 and 1). For example, the belief about the proportion of Nother survey-participants

who participate in a protest is a probability distribution over the N+ 1 possible values

{0,1/N, 2/N, ..1}. Yet, for tractability, researchers often have to elicit point-responses in

their experimental design, and interpret those as a coarse measure of beliefs. This is the

case in the papers cited above, as well as many others.2

In this paper, we discuss the relation between subjects’ point-responses and their underlying

belief distribution, and how this mapping depends on incentives offered to the subject. In

particular, we compare two popular belief elicitation schemes that seem superﬁcially simi-

lar, but, as we show in Section 2, incentivize different best responses in belief reporting. We

then show the empirical consequences of such differences, which can include identifying

effects with opposite signs to the true ones, or a lack of identiﬁcation altogether.

The ﬁrst scheme we consider rewards subjects for correct guesses within an error band

of the true value: for instance, “Please guess x∈[0,1]. If your guess is within ∆percentage

points of x∈[0,1] you will earn a bonus payment of 1 currency unit.” Recent examples of

belief elicitation using this scheme include Cantoni et al. (2019); Chen and Yang (2019);

Bursztyn et al. (2020), among others. In Section 2.1, we prove that subjects’ best response

to these incentives is to report the (approximate) mode of the true distribution of x(hence-

forth, modal beliefs). The second one, which we recommend to practitioners who wish to

elicit the mean of the belief distribution over x(mean beliefs, henceforth), rewards sub-

jects A−B(x−r)2for reporting r, where A, B are constants. Such an incentive scheme

indeed induces proﬁt-maximizing subjects to report their mean beliefs as a best response.

The difference in elicitation designs is subtle: rewards under both schemes are weakly

increasing in accuracy. But they induce very different mappings from the true belief distri-

bution (over x) to the optimal report, as mean and mode do not generally coincide. For

1Such examples go beyond political economy. In another recent example, Bursztyn et al. (2020) evaluate

whether Saudi husbands are more likely to support women working outside the home if they discover that

other husbands do so too.

2Kendall et al. (2015); Cruz et al. (2020) are some exceptions in the experimental political economy context.

FIGURE 1. Simple Illustration that Means and Modes are Updated in Oppo-

site Directions

We present an example of how means and modes can be updated in opposite directions, implying that

different belief elicitation schemes may provide different signs for identiﬁed effects. In the example,

beliefs follow x∼Beta(1.5,4), where the parameters are calibrated to the empirical application dis-

cussed in Section 4.3. The mode (dotted red line) is equal to 0.142, while the mean (solid red) equals

0.273. An information intervention (black dotted line) that reveals the true state to be 0.17 is intro-

duced to subjects. Beliefs are updated towards 0.17, which implies that the mode increases, but the

mean decreases (shown by the black arrows).

the sake of completeness, we also discuss a third incentive scheme, which rewards subjects

based on the absolute distance between their report and the true value. This incentivizes

reporting the median of the true distribution instead.

We show that these differences can be very consequential when it comes to testing a

theoretical prediction. Not only are modes, means and medians generally different, but

they may have very different properties following an information intervention. This can

be visualized within a simple example, shown in Figure 1. In this example, beliefs about

a random variable, x, are assumed to follow a ﬂexible continuous distribution, which has

been calibrated to our empirical application (described below). In Figure 1, the mode

is given by 0.142, while the mean is 0.273. Now, we assume there is an experimental

information intervention, like those in the papers cited above. In particular, the treatment

reveals that xis equal to 0.17, akin to Cantoni et al. (2019). Beliefs will then update

towards 0.17, implying the mode will increase towards 0.17, while the mean will decrease

accordingly, as illustrated by the arrows in the ﬁgure.

Hence, a researcher who uses incentives to measure modal beliefs but interprets the data

as mean beliefs, will have identiﬁed a very different measure than they intended to. And

using this measure may lead to very different conclusions while testing a theory-derived

hypothesis. In fact, as the example above shows, researchers who interpret modal and

mean beliefs interchangeably, may (i) fail to identify the theoretical parameter of interest,

or may (ii) identify their opposite sign (if the parameter is deﬁned by how changes in mean

beliefs relate to changes in actions). We formalize these points in Section 3.

As an empirical application, we revisit Cantoni et al. (2019)’s experiment on political

protests in Hong Kong.3Cantoni et al. (2019) conduct a cleverly designed ﬁeld exper-

iment in the backdrop of an anti-authoritarian protest in Hong Kong. They show that

potential protesters who increased their reported beliefs in favor of others being more

likely to protest, became less likely to protest themselves. Interpreting the reported beliefs

as mean beliefs, Cantoni et al. (2019) conclude that “speciﬁcally, our ﬁndings reject many

recent models that assume only the possibility of strategic complementarity in the protest

decision.” (p.1072) That is, they interpret their evidence as contradicting the widespread

assumption that political protests are strategic complements (i.e., when a citizen believes

that more peers are likely to protest, she should become more likely to protest herself, see

Gehlbach et al. (2016)).

However, we argue that the empirical results in Cantoni et al. (2019) do not necessar-

ily reject strategic complementarity (SC) or the many theoretical models that assume it.

Rather, we show that Cantoni et al. (2019)’s evidence can be completely consistent with

SC using a two-fold argument. First, Cantoni et al. (2019)’s experimental design uses the

ﬁrst belief-elicitation scheme described above, thereby incentivizing subjects to report their

modal beliefs, but not their mean beliefs. Thus, their belief data does not necessarily reveal

mean beliefs. Second, we use a stylized theoretical model and closely derived statistical

results to show that means and modes can be updated in opposite directions in their set-up,

as shown in Figure 1. Therefore, their estimates of a negative covariance between belief

updating about others’ protest attendance and one’s own attendance may be ﬂipped.

The distinction borne in our discussion is also empirically meaningful. We perform a

numerical exercise where we ﬁt ﬂexible distributions of individual beliefs to the distribu-

tion of observed prior and posterior reports in the Cantoni et al. (2019) dataset, assuming

subjects were reporting modal beliefs about others’ participation. Based on the estimated

distribution of beliefs, we ﬁnd that over a third of subjects would have updated their mean

belief in the opposite direction of their reported belief. Therefore, how one interprets

“elicited beliefs” determines what the data of Cantoni et al. (2019) implies about strategic

complementarity.

To be clear, our analysis does not aim to refute or reverse any regression results from

Cantoni et al. (2019). Rather, we use it as a prominent example of how elicitation schemes

3We also discuss Hager et al. (2022a); Jarke-Neuert et al. (2021); Hager et al. (2022b) that use a similar

research design and where our insights also apply.

inﬂuence the interpretation of economic data: The scheme used in Cantoni et al. (2019) al-

lows alternative interpretations of their results, and such interpretations are meaningful to

what we can learn about the empirical validity of strategic complementarity. Such alterna-

tive explanations could have been completely avoided with the alternative mean-eliciting

scheme that we propose. None of what we say precludes the possibility that subjects have

unimodal symmetric beliefs and, thus, their mean and modal beliefs are identical, thereby

preserving their original interpretation in Cantoni et al. (2019). Instead, we think of our

results as clarifying the important role that this implicit assumption (unimodal symmet-

ric beliefs) plays in linking the regression results to the ﬁnal conclusion, both in their

experiment and in others, emphasizing the importance of the design of belief elicitation

procedures per se.

2. Belief Elicitation

Suppose subjects believe that some outcome x∈[0,1] is distributed according to f, a

probability density function over [0,1]. In Cantoni et al. (2019) and Hager et al. (2022a),

for instance, xis the percentage of the participants from the study who plan to participate

in a political protest. In Bursztyn et al. (2020), it is the percentage of Saudi husbands

who believe women should be allowed to work outside the home. Suppose researchers are

interested in eliciting a single value that is in some way representative of the distribution

2.1. Elicitation Schemes that Incentivize Reporting the Mode

Consider the following general incentive system:

“Please guess x∈[0,1]. If your guess is within ∆percentage points of

x∈[0,1] you will earn a bonus payment of 1 currency unit.”

Cantoni et al. (2019) use a special case of this with ∆=2, and where 1 currency unit was

10 HKD. Similarly, Chen and Yang (2019) sets ∆ = 0.1and the currency unit was RMB 5.

Meanwhile Bursztyn et al. (2020) uses some ∆close to 0 with strategic uncertainty about

its value and US$20 as a currency unit.4

There exists simple sufﬁcient conditions on funder which, under this elicitation scheme,

proﬁt maximizing subjects should report the mode exactly, or approximately with an error

4Technically, the incentives in Bursztyn et al. (2020) award the participant “who guesses most accurately”.

This could induce strategic guessing, where a subject anticipates other subjects’ guesses when choosing their

report. However, with a large enough population of independent subjects (1500 in their sample), payment

should reﬂect being arbitrarily close to the true values (∆→0).

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

CHOOSINGTHEBESTINCENTIVESFORBELIEFELICITATIONWITHANAPPLICATIONTOPOLITICALPROTESTSNathanCanenandAnujitChakrabortyABSTRACT.Manyexperimentselicitsubjects'priorandposteriorbeliefsaboutarandomvariabletoassesshowinformationaffectsone'sownactions.However,beliefsaremulti-dimensionalobjects,andexperimenterso...

展开>> 收起<<

CHOOSING THEBESTINCENTIVES FOR BELIEF ELICITATION WITH AN APPLICATION TO POLITICAL PROTESTS Nathan Canen and Anujit Chakraborty.pdf

共24页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

CHOOSING THEBESTINCENTIVES FOR BELIEF ELICITATION WITH AN APPLICATION TO POLITICAL PROTESTS Nathan Canen and Anujit Chakraborty

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: