Lets assume there is a deck of 52 cards with 4 suits face down.
In each round I turn over the top card from the deck, this card changes my known information and changes the probability of the suit of the last card in the deck.
I know from experience that early cards in the deck change the probability of the suit of the last card incrementally but at the end they change the probability more dramatically, for example if the first card is a heart it changes the probability of the last card being a heart from 13/52 to 12/51 whereas if its one of the last cards it can change it from 2/5 to 2/4.
I was wondering if there was a way to calculate for an unknown deck, the variance (or any other metric) in probability of each card against the probability of the suit of the final card.
Thanks!
Let $X_i$ denote the suit of the $i$-th card from the top of the deck; so $X_i$ takes values in the set $\{\text{Clubs, Hearts, Spades, Diamonds}\}$. So you are interested in the variable $X_{52}$.
In fact, before we start dealing any cards it is the case that for any $1 \leq i,j \leq 52$ $$ \mathbf P[X_i = s] = \mathbf P[X_j = s],$$ since (for instance) it is no more likely that the 1st card is a Club than the 34th card is. Similarly if we suppose that we have already dealt the first $n$ cards, so that $X_1, \ldots X_n$ are known then it is similarly the case for $n < i,j \leq 52$
$$ \mathbf P[X_i = s \, | \, X_1, \ldots, X_n] = \mathbf P[X_j = s \, | \, X_1, \ldots, X_n],$$
where we used the standard notation for conditional probability; i.e. the probability that $X_i = s$ having observed cards $X_1, \ldots X_n$.
Further, the exact order of the cards observed does not impact the probability that the last card is a given suit: only the total number of each suit observed. So if we define each of $C_n, H_n, S_n, D_n$ to be the total number of Clubs, Hearts, Spades, Diamonds seen after $n$ cards, e.g.
$$ C_n = \# \{ 1 \leq i \leq n \, \colon \, X_i = \text{Clubs} \}. $$
Then for instance, in the case of clubs we have
\begin{align*} \mathbf P[X_i = \text{Clubs} \, | \, X_1, \ldots, X_n] &= \mathbf P[X_i = s \, | \, C_n,\,H_n,\,S_n,\, D_n]\\ & = \mathbf P[X_i = s \, | \, C_n\,]\\ & = \frac{13 - C_n}{52 - n} \end{align*}
Note that this matches the formula you provided in your question, since if we have turned over just one card, and it was a heart then
$$ \mathbf P[X_{52} = \text{Hearts} | X_1 = \text{Hearts}] = \mathbf{P}[X_{52} = \text{Hearts} | H_1 = 1] = \frac{13 - 1}{52 - 1} = \frac{12}{52}.$$
Now since the variables $X_i$ are not numeric, we cannot directly define expectation and variance of them; for instance, the mean of $X_i$ does not make sense because $X_i$ is not a number.
One way to answer this question is instead to define for each suit $s \in \{\text{Clubs, Hearts, Spades, Diamonds}\}$ and each $1 \leq i \leq52$ the variable
$$Y_{i,s} = \begin{cases} 1 & \text{if $C_i = s$,} \\ 0 & \text{else.} \end{cases} $$ Now $Y_{i,s}$ is a numeric variable, and we have that $Y_{i,s} = 1$ if and only if $C_i = s$. Further we have that the expectation of $Y_{i,s}$ satisfies $$\mathbf E[Y_{i,s}] = \mathbf P[Y_{i,s} = 1] = \mathbf P[C_i = s].$$
Whilst the variance is
\begin{align*}\text{Var}(Y_{i,s}) &= \mathbf E [ Y_{i,s}^2] - \mathbf E[Y_{i,s}]^2 \\ & = \mathbf E [ Y_{i,s}] - \mathbf E[Y_{i,s}]^2 \\ & = \mathbf E [ Y_{i,s}^2] \left( 1 - \mathbf E[Y_{i,s}] \right)\\ & = \mathbf P[C_i = s] \left( 1 - \mathbf P[C_i = s] \right) \end{align*} Similarly, if we have already observed $X_1,\ldots, X_n$ then we have the equivalent formulae
\begin{align*} \mathbf{E}[Y_{i,s} \, | \, X_1, \ldots, X_n] & = \mathbf{P}[X_i = s \, | \, X_1,\ldots, X_n] \\ \text{Var}(Y_{i,s} \, | \, X_1 \ldots, X_n ) & = \mathbf{P}[X_i = s \, | \, X_1,\ldots, X_n] \left( 1 - \mathbf{P}[X_i = s \, | \, X_1,\ldots, X_n] \right) \end{align*}
For instance before any cards are drawn, we have for all suits $s$, and all $1 \leq i \leq 52$ \begin{align*} \mathbf{E}[Y_{i,s}] &= \frac{13}{52}, \\ \text{Var}(Y_{i,s}) & = \frac{13}{52} \left(1- \frac{13}{52}\right). \end{align*}
And if we consider the case that $n$ cards have been dealt and $s = \text{Hearts}$ then
\begin{align*} \mathbf{E}[Y_{i,s} \, | \, X_1,\ldots, X_n] &= \mathbf{E}[Y_{i,s} \, | H_n ] \\ & = \frac{13 - H_n}{52 - n}, \\ \text{Var}(Y_{i,s}| \, X_1,\ldots, X_n) & = \text{Var}(Y_{i,s}| \,H_n) \\ & = \frac{13 - H_n}{52 - n} \left(1- \frac{13 - H_n}{52 - n}\right). \end{align*}
Finally, as mentioned in one of the comments, the standard way to consider variance of a categorical object, without passing to the indicator function is in terms of the entropy. We would define the entropy of the $i$-th card to be
$$\mathcal H(X_i) = - \sum_{s} \mathbf P[X_i = s] \log_2 \mathbf P[X_i = s],$$ and using the formulae from before we see that this is (before dealing any cards)
$$ \mathcal H(X_i) = - \sum_s \frac{13}{52} \log_2 \frac{13}{52} = - \log_2 \frac{13}{52}.$$
After having observed $n$ cards, for $n < i \leq 52$ this formula becomes
\begin{align*} \mathcal H(X_i \, | \, X_1,\ldots, X_n) & = \mathcal H(X_i \, | \, C_n, \, H_n,\, S_n, \, D_n) \\ & = \frac{13 - C_n}{52 - n} \log_2 \frac{13 - C_n}{52 - n} + \frac{13 - H_n}{52 - n} \log_2 \frac{13 - H_n}{52 - n} + \\ & \qquad \qquad \frac{13 - S_n}{52 - n} \log_2 \frac{13 - S_n}{52 - n} + \frac{13 - D_n}{52 - n} \log_2 \frac{13 - D_n}{52 - n} \end{align*}
This is rather complex in the notation I have used thus far; if we use the simpler notation of $N_n(s) = \#\{1 \leq i \leq n \, \colon \, X_i = s\}$ to denote the number of occurences of suit $s$ this simplifies to
$$ \mathcal H(X_i \, | \, X_1,\ldots, X_n) = \mathcal H(X_i \, | N_n(s) ) = \sum_s \frac{13 - N_n(s)}{52 - n} \log_2 \frac{13 - N_n(s)}{52 - n} $$
Much like variance, when the `randomness' decreases, so does entropy. For instance one can show that $$\mathcal H(X_i) > \mathcal H(X_i \, | \, X_1 ) > \cdots > \mathcal H(X_{52} \, | \, X_1,\ldots X_{51}),$$ and moreover that
$$\mathcal H(X_{52} \, | \ X_1, \ldots X_{51}) = 0,$$ i.e. that when there is only one card left, we know its suit therefore there is no randomness, and the entropy is $0$.