Imagine we toss a coin 10 times and we get 6 heads.
How would we go about:
These were actually interview questions. I thought I could start by using Bayes theorem assuming that the coin being biased had a probability 'p'... but that wasn't really taking me anywhere...
How should I go about solving these questions?
On
If you want a Bayesian approach, you will want a prior distribution for the probability $p$ of heads. With a Bernouilli or binomial random variable, the conjugate family (whose main merit is that it is easiest to work with) is the Beta distribution with parameters $\alpha$ and $\beta$
Seeing $h$ heads and $t$ tails, i.e. a likelihood proportional to $p^h(1-p)^t$, will give a posterior distribution for $p$ which is also a Beta distribution but with parameters $\alpha+h$ and $\beta+t$. This posterior distribution will have a mean of $\dfrac{\alpha+h}{\alpha+h+\beta+t}$, a mode of $\dfrac{\alpha+h-1}{\alpha+h+\beta+t-2}$, and a standard deviation of $\sqrt{\dfrac{(\alpha+h)(\beta+t)}{(\alpha+h+\beta+t)^2(\alpha+h+\beta+t+1)}}$
Common choices for the prior are $\alpha=\beta=1$ (a uniform prior), $\alpha=\beta=0$ (an improper Haldane prior), and $\alpha=\beta=\frac12$ (a Jeffreys prior)
For example, starting with $\alpha=\beta=\frac12$ and your observation of $h=6,t=4$ would give a posterior distribution for $p$ with mean about $0.59$, mode about $0.61$ (compare these to the naive estimate of $\frac6{10}=0.6$) and standard deviation about $0.14$
The more times you repeat the experiment the better the estimate of the true probability. This is the Strong Law of Large Numbers https://en.m.wikipedia.org/wiki/Law_of_large_numbers