Definition of confidence interval

Question

Consider the statistical model $\left( E, \{\Bbb P_ \theta\}_{\theta \in \Theta}\right)$(with $\Theta \subseteq \Bbb R$) and a random i.i.d sample $X_1, \ldots, X_n$ from the true distribution $P_{\bar{\theta}}.$ Note that we fix the true parameter $\bar{\theta}$ to differentiate it from the generic $\theta$ that is used for denoting the elements in $\Theta.$ Let $\alpha \in (0, 1).$ Which of the following definitions of a confidence interval of level $1 - \alpha$ is the correct one? Could you point me to a reference on the correct definition?

A). An interval $\mathcal{I}_n$ whose boundaries do not depend on $\bar{\theta}$ (but are possibly a function of the sample) such that $\Bbb P_{\theta}(\mathcal{I}_n \ni \bar{\theta}) \geq 1 - \alpha \;\;\forall \;\theta \in \Theta.$

B). An interval $\mathcal{I}_n$ whose boundaries do not depend on $\bar{\theta}$ (but are possibly a function of the sample) such that $\Bbb P_{\theta}(\mathcal{I}_n \ni \theta) \geq 1 - \alpha \;\;\forall \;\theta \in \Theta.$

C). An interval $\mathcal{I}_n$ whose boundaries do not depend on $\bar{\theta}$ (but are possibly a function of the sample) such that $\Bbb P_{\bar{\theta}}(\mathcal{I}_n \ni \bar{\theta}) \geq 1 - \alpha.$

Answer 1

From Shao's Mathematical Statistics p. 471:

(...) $X=(X_1,...,X_n)$ denotes a sample from a population $P \in \mathcal{P}$; $\theta=\theta(P)$ denotes a functional from $\mathcal{P}$ to $\Theta\subset \mathcal{R}^k$ for a fixed integer $k$; and $C(X)$ denotes a confidence set for $\theta$, a set in $\mathcal{B}_\Theta$ (the class of Borel sets on $\Theta$). We adopt the basic concepts of confidence sets introduced in §2.4.3. In particular, $\inf_{P\in \mathcal{P}}P(\theta \in C(X))$ is the confidence coefficient of $C(X)$, and if the confidence coefficient of $C(X)$ is $\geq 1-\alpha$ for fixed $\alpha \in (0,1)$, then we say that $C(X)$ has significance level $1-\alpha$ or $C(X)$ is a level $1-\alpha$ confidence set.

and §2.4.3 at p. 129, where notation differs slightly:

Consider a real-valued $\vartheta$. If $C(X)=[\underline{\vartheta}(X),\overline{\vartheta}(X)]$ for a pair of real valued statistics $\underline{\vartheta}$ and $\overline{\vartheta}$, then $C(X)$ is called a confidence interval for $\vartheta$.

Also, from p. 129 again:

(...) $C(X)\in \mathcal{B}^k_\Theta$ depending only on the sample $X$.

So $C(X)$ is a confidence interval if $C(X)=[\underline{\vartheta}(X),\overline{\vartheta}(X)]$ and $$\inf_{P \in \mathcal{P}}P(\theta\in [\underline{\vartheta}(X),\overline{\vartheta}(X)])\geq 1-\alpha$$ Since $\theta: \mathcal{P}\to \Theta$ is a functional of $P$, the answer seems to be (B).

Answer 2

After a cursory scan of some textbooks (Lehmann and Romano, "Testing Statistical Hypotheses" sec. 5.4; Keener, "Theoretical Statistics" sec. 9.4; or Wasserman "All of Statistics" sec 6.3.2), the most common definition seems to be (B).

However, the intuitive definition for a confidence interval is an interval that contains the true parameter (ie $\bar{\theta}$) with a given probability, which suggests that (C) is the more appropriate definition, so why is it less common?

As you say, (B) implies (C). But if we are going to prove that (C) holds without knowing $\bar{\theta}$, the only way of doing this is to prove that it holds regardless of the true value of $\bar{\theta}$. But that's exactly what (B) means! Therefore, we might as well take (B) as the definition in the first place; despite being seemingly more restrictive, in practice we don't gain anything from a looser definition.

Answer 3

The difference between B and C is, I believe, a linguistic distinction without a difference.

From the outset, we can either declare that there is a "true" value $\overline{\theta}$ of the parameter, or we can decline to do that. In the latter case, when we decline to declare that there is a "true" value, the meaning of the model is that the different values of $\theta$ in $\Theta$ exist in parallel universes, and in the $\theta$ universe, that value of $\theta$ is the true value in that universe. In this case, we're not saying that there is no true value of the parameter. We're effectively saying that each value is/could be the true value. Informally, this is equivalent to saying that there is a "true" value $\overline{\theta}$, but we don't know what it is and all of our theory/definitions/formulas, etc., will remain as stated regardless of which value of $\overline{\theta}$ is the true one.

The expression in C is consistent with the linguistic convention of declaring that there is a "true" value $\overline{\theta}$, and the expression in B is consistent with the linguistic convention of declining to do so. There is an explicit "$\forall \theta\in \Theta$" in B, and there is an implicit "$\forall \overline{\theta}\in \Theta$", because C is intended to hold regardless of which value of $\overline{\theta}$ is the "true" value.

As a thought experiment to try to understand whether there really is a fundamental difference between B and C, I would ask: In estimation, $\overline{\theta}$ is the true value, why would you ever compute/care about $\mathbb{P}_\theta$ for any $\theta$ other than $\overline{\theta}$? For an event $A$, $\mathbb{P}_\theta(A)$ exactly refers to the probability of the event $A$ in that universe in which $\theta$ is the true value of the parameter.