
Algorithmic Fairness. A multitude of formal fairness
definitions have been put forward in the literature (Verma
and Rubin 2018). Examples include statistical parity (Dwork
et al. 2012), predictive parity (Chouldechova 2017), equal-
ized odds, equality of opportunity (Hardt, Price, and Srebro
2016), and individual fairness (Dwork et al. 2012). How-
ever, they are still a topic of discussion, for instance, be-
cause these definitions are known to be incompatible (Klein-
berg, Mullainathan, and Raghavan 2016;Lipton, McAuley,
and Chouldechova 2018). Additionally, there are a several
definitions that rely on causal mechanisms to assess fair-
ness, e.g., counterfactual fairness (Kusner et al. 2017), and
the notion of unresolved discrimination (Kilbertus et al.
2017). While causal approaches to fairness might be prefer-
able, they require information about the causal structure of
the data generating process. Moreover, it has recently been
shown that causal definitions may lead to adverse conse-
quences, such as lower diversity (Nilforoshan et al. 2022).
We discuss how existing fairness definitions could possibly
be applied to the setting with optional features, but we find
that none of the fairness definitions aligns with our desider-
ata theoretically and experimentally (see Appendix A.2).
Strategic Classification. In an even broader context, this
work also relates to the field of strategic classification (Hardt
et al. 2016). However, it is worth noting that in strategic
classification research, the focus primarily revolves around
users strategically manipulating their features for optimal
outcomes, which may also involve information withholding
(Krishnaswamy et al. 2021). In contrast to our work, privacy
concerns are neglected in this research stream. As far as we
are aware, there are no prior works on the specific problem
of balancing the interests of all three groups of stakeholders
(the non-sharers, sharers, and the decision makers).
3 Problem Formulation
3.1 Formalization and Notation
In this work, each data instance contains a realization of a
number of base features b∈ Xb, where Xb⊆Rnis the
space of the base features. Furthermore, let there be some
optional information z∈ Xz, where Xz⊆Ris the value
space of the optional feature.1It is the users’ choice to de-
cide if they want to disclose zto the system, which results
in an availability variable a∈ {0,1}. Accordingly, only
imputed samples z∗={zif a=1,else N/A}are observed,
where a value of N/A indicates that a user did not reveal the
optional information, e.g., did not use the companion app.
In summary, the data observations are tuples x= (b, a, z∗)
that reside in X=Xb×{0,1}×(Xz∪{N/A}). Each train-
ing sample comes with a label y∈ Y. Further, there is a
data generating distribution pwith support X × Y and we
have access to an i.i.d. training sample (x, y)∼p. Figure 2
shows such a data sample. We denote the random variables
for the respective quantities by B, A, Z, Z∗, Y . The label is
probabilistically determined through the base features Band
the hidden feature Zbut the sharing decision does not influ-
1We extend our definitions to integrate multiple optional fea-
tures a later section.
base features bopt. feat. z∗alabel y
state plan fitness scoreavail.treatment costs
New South Wales basic 87 % 1 3k$
Queensland gold N/A 0 17k$
New South Wales basic 92 % 1 5k$
New South Wales basic N/A 0 64k$
Victoria premium 56 % 1 22k$
Figure 2: Samples for the insurance use-case. We have two
base features band one optional feature z∗, which either
takes an observed value z, or it takes a value of N/A if un-
observed. The variable a∈ {0,1}indicates the availability
of the feature. The goal is to predict the label y.
ence the true label for a given B, Z, such that Y⊥⊥ A|B, Z.
In many applications, the goal is to find a function f:
X → Y that models the observed data. In particular, f:
X → [0,1] may predict a probability of a positive outcome
or f:X → Rmay return a numerical score. The test
data for which the model will be used come from the same
distribution p, though with the label yunobserved, and we
suppose that the information provided is always correct. We
consider a convex loss function L:Y ×Y → R, e.g., mean-
squared-error (MSE) or binary cross entropy (BCE), for
which we minimize the expected loss for a sample from the
data distribution. For instance, using the common MSE loss
L(f(x), y)=(f(x)−y)2, an optimal predictor is given by
f∗
L(x) = arg minf(x)Ep(Y|x)(f(x)−Y)2=E[Y|x],
the conditional expectation. However, this notion can be
generalized to other loss functions: An optimal predictor
f∗
L(x)for the loss function Lfulfills ∀x:
f∗
L(x) = FL
p[Y|x]:= arg min
f(x)
Ep(Y|x)[L(f(x), Y )] .(1)
We use FL[Y|x]to denote a generalized expected value that
minimizes the expected loss conditioned on x. To ease our
derivations, we suppose this minimum to be unique and fi-
nite. Intuitively, it represents the best guess of Ygiven x. For
the MSE-Loss, FLis equivalent to the expectation operator
E. In the following statements, the reader may thus mentally
replace FLwith an expectation Ewithout further ramifica-
tions in order to get the high level intuition. Finally, we in-
troduce two key terms, namely, base feature model and full
feature model. The former refers to a model trained on the
base features only, while the latter refers to a model trained
on all features where some strategy is used to replace un-
available feature values. Typically these strategies are called
imputation and replace unavailable values by zeros, a fea-
ture’s mean or median (Emmanuel et al. 2021).
3.2 Desiderata
Our goal is to learn models f:X → Y that comply with
the desideratum of Availability Inference Restriction, which
we briefly introduced in Section 1, to protect the interests
of the non-sharers. Under this constraint, the model should
provide the best predictive performance to reflect the need