I just read this interesting xkcd strip:

At first I thought it was funny, but as I got to ruminate a little over it, I was surprised to be unable to find an answer. As Karolis Juodelė pointed out, the probability is ε, as there is an infinite number of bases containing 1 and 10.
However, to get a finite answer, we can modify the puzzle like this:
On a scale of 1 to 10, how likely is it that this question is using binary vs. decimal?
So my question is: How should I solve this puzzle? Is there a correct answer at all? Is this what we call a self-reference paradox, like Multiple-choice question about the probability of a random answer to itself being correct?
(If I understand correctly, $10$ means certainly in binary and $1$ means certainly in decimal)
To answer your modified question, if I were to think like in your last comment (2014-01-11 20:48:42Z), then I would answer:
But for me, I choose to answer