Interpreting $P(\alpha|\text{data})\propto P(\text {data} | \alpha)\cdot P(\alpha)$

Question

In the context of posterior and prior probabilities, one has $P(\alpha|\text{data})\propto P(\text {data} | \alpha)\cdot P(\alpha)$. What confuses me here is that probability is defined for events, and $\alpha$ is not an event, it is a parameter. But then how to interpret $P(\alpha)$ and $P(\alpha|\text{data})$ if $\alpha$ is not an event?

Answer 1

I'm not familiar with using capital $P$ in this context, which can be confusing. Typically, $P(\alpha)$ and $P(\alpha|\text{data})$ are pdfs/pmfs of the parameters $\alpha$ and $\alpha|\text{data}$.

For example, suppose $\alpha\sim\text{Beta}(\kappa,\lambda)$ and $N\sim \text{Binomial}(n,\alpha)$, then in your notation, $$P(\alpha)=\frac{\alpha^{\kappa-1}(1-\alpha)^{\lambda-1}}{B(\kappa,\lambda)}.$$ I would prefer to use the notation $f(\alpha)$ instead of $P(\alpha)$, because $P$ can look like a probability, which, as you said, should be probability of an event. I've also seen authors use $\pi(\alpha)$, where $\pi$ is for prior.

Similarly, if the data we observe is that $N=k$, then $\alpha|\text{data}\sim\text{Beta}(\kappa+k,\lambda+n-k)$, and $$P(\alpha|\text{data})\propto\alpha^{\kappa+k-1}(1-\alpha)^{\lambda+n-k-1}.$$

In this context, $\text{data}$ is often also an outcome, not an event, and $P(\text{data}|\alpha)$ is the value of a pdf/pmf.