somewhat different from that given above, and less succinct.)
3.4 The essential distinction between the above presentation and that of Chalmers is that he makes
use (in the first (2) of his Section 2) of the stronger conditional assumption "I know that I am F",
rather than merely "I am F", the latter being all that I need for the above. Thus, if we accept the
validity of the above argument, the conclusion is considerably stronger than the "strong" conclusion
that Chalmers draws ("threatening to the prospects of AI") to the effect that it "would rule out even
the possibility that we could empirically discover that we were identical to some system F".
3.5 In fact, it was this stronger version (A) that I presented in Shadows, from which we would
conclude that we cannot be identical to any knowable (Gödelizable) system F whatever, whether we
might empirically come to believe in it or not! I am sure that this stronger conclusion would provide
an even greater motivation for people (whether AI supporters or not) to find a flaw in the argument.
So let me address the particular objection that Chalmers (and, in effect, also McCullough) raises
against it.
3.6 The problem, according to Chalmers, is that it is contradictory to "know that we are sound".
Accordingly, he argues, it would be invalid to deduce the soundness of F, let alone that of F', from
the assumption "I am F". On the face of it, to a mathematician, this seems an unlikely let-out, since
in all the above discussions we are referring simply to the notion of mathematical proof. Moreover,
the "I" in the above discussion refers to an idealized human mathematician. (The problems that this
notion raises, such as those referred to by McDermott, are not my concern at the moment. I shall
return to such matters later; cf. Section 6.) Suppose that F indeed represents the totality of the
procedures of mathematical proof that are in principle humanly accessible. Suppose, also, that we
happen to come across F and actually entertain this possibility that we might "be" F, in this sense
(without actually knowing, for sure, whether or not we are indeed F). Then, under the assumption
that it is F that encapsulates all the procedures of valid mathematical proof, we must surely
conclude that F is sound. The whole point of the procedures of mathematical proof is that they instil
belief. And the whole point of the Gödel argument, as I have been employing it, is that a belief in
the conclusions that can be obtained using some system H entails, also, a belief in the soundness
and consistency of that system, together with a belief (for a Gödelizable H) that this consistency
cannot be derived using H alone.
3.7 This notwithstanding, Chalmers and McCullough argue for an inconsistency of the very notion
of a "belief system" (which, as I have pointed out above, simply means a system of procedures for
mathematical proof) which can believe in itself (which means that mathematicians actually trust
their proof procedures). In fact, this conclusion of inconsistency is far too drastic, as I shall show in
a moment. The key issue is not that belief systems are inconsistent, or incapable of trusting
themselves, but that they must be restricted as to what kind of assertion they are competent to
address.
3.8 To show that "a belief system which believes in itself" need not be inconsistent, consider the
following. We shall be concerned just with P-sentences (which, we recall, are assertions that
specified Turing machine actions do not halt). The belief system B, in question, is simply the one
which "believes" (and is prepared to assert as "unassailably perceived") a P-sentence S if and only
if S happens to be true. B is not allowed to "output" anything other than a decision as to whether or
not a suggested P-sentence is true or false - or else it may prattle on, as is its whim, generating P-
sentences together with their correct truth values. However, as part of its internal musings, it is
allowed to contemplate other kinds of thing, such as the fact that it does, indeed, produce only
truths in its decisions about P-sentences. Of course, B is not a computational system - it is a Turing
oracle system, as far as its output is concerned - but that should not matter to the argument. Is there