Suppose there is an urn with 100 balls, of two colors, say white and black. Let $p$ be the probability of drawing a white ball. You draw one ball, replacing after the draw. After 100 draws, each with replacement, you observe 100 white balls. What is the value of $p$?
Since there is replacement, this seems to be a binomial experiment where $X$ counts the number of white balls out of $n=100$ draws. In other words, $X\sim Bin(n,p)$, so $Pr(X=100) = p^{100}$.
If we were testing some null hypothesis about the value of $p$, then we can use the observation to accept or reject the null. But without a null hypothesis on the value of $p$, perhaps we need to compute a Bayesian posterior probability distribution?
I'm also a little baffled by the fact that if $p=0.99$, $P(X=100) = 0.99^{100} = 0.366$, which seems really small. What is the best way to approach this problem?
If your prior for the parameter $p$ has a density $f$, then Bayes' rule says that the posterior density for $p$ after observing $X=100$ is
$$f(q \mid X=100)=\frac{P(X=100 \mid p=q) f(q)}{\int_0^1 P(X=100 \mid p=s) f(s) ds}.$$
In other words you multiply your prior by the likelihood function $P(X=100 \mid p=q)=q^{100}$ and normalize. In the case of the uninformed prior you take $f \equiv 1$ (the uniform distribution on $[0,1]$), and so the result is $101q^{100}$. Unsurprisingly, this function is quite sharply peaked near $q=1$. In particular its expected value is $\frac{101}{102}$ (a special case of the rule of succession mentioned in joriki's comment).