I am trying to estimate the probability of a biased coin coming up heads and how confident I can be of this estimate. Looking at this post here, and assuming I have observed $ X = 7 $ heads in $ N = 8 $ trials, I have made the following calculations.
$\hat{p} = \dfrac{1}{n}\displaystyle\sum_{i = 1}^{n}X_i = 0.875$
$\text{Var}[\hat{p}] = \dfrac{p(1-p)}{n} = 0.0137$
I then tried to estimate the 95% confidence interval using a T distribution as sample size is small.
\begin{align*} CI &= \hat{p} \pm t\frac{\sqrt{Var[\hat{p}]}}{\sqrt{N}}\\ &= 0.875 \pm 1.895 \frac{0.227}{2.83}\\ &= (0.797, 0.908) \end{align*}
However now I am trying to find the value of $\hat{p}$ where I would reject the null hypothesis $H_0: p \le \hat{p}$ at a 95% confidence level and also the p-value of my estimate of $\hat{p} = 0.875$. My question is how this is related to the confidence interval (if at all) and how I can determine this value. My initial intuition is that I could determine this by solving for $0.05 \geq (p^X)\cdot(1-p)^{N-X}\cdot {N \choose X}$ but I am not sure how I could solve this equation either.
Your hypothesis $$H_0 : p \le \hat p$$
makes no sense. The hypothesis needs to be a statement about the parameter that does not depend on the test statistic. For example, one possible hypothesis is to ask
$$H_0 : p = 0.5 \quad \text{vs.} \quad H_1 : p \ne 0.5.$$
This hypothesis tests if the coin is unfair (either biased toward heads or toward tails). Or, you can ask
$$H_0 : p \le 0.5 \quad \text{vs.} \quad H_1 : p > 0.5.$$
If $p$ represents the probability of heads, then this hypothesis asks whether the coin is biased towards heads. Also, there is nothing special about $0.5$; we could ask
$$H_0 : p = 0.75 \quad \text{vs.} \quad H_1 : p \ne 0.75,$$
for instance. In each case, the test statistic is the same but the rejection region for the test will be different.
Allow me to illustrate with the first hypothesis above. Suppose $n = 8$ and we observed $X = 7$ heads. Then, under the null hypothesis that the coin is fair, $$X \mid H_0 \sim \operatorname{Binomial}(n = 8, p = 0.5).$$ We then compute the left- and right-tail probabilities
$$\begin{array}{c|c|c|c} x & \Pr[X = x] & \Pr[X \le x] & \Pr[X \ge x] \\ \hline 0 & \frac{1}{256} & \frac{1}{256} & 1 \\ 1 & \frac{1}{32} & \frac{9}{256} & \frac{255}{256} \\ 2 & \frac{7}{64} & \frac{37}{256} & \frac{247}{256} \\ 3 & \frac{7}{32} & \frac{93}{256} & \frac{219}{256} \\ 4 & \frac{35}{128} & \frac{163}{256} & \frac{163}{256} \\ 5 & \frac{7}{32} & \frac{219}{256} & \frac{93}{256} \\ 6 & \frac{7}{64} & \frac{247}{256} & \frac{37}{256} \\ 7 & \frac{1}{32} & \frac{255}{256} & \frac{9}{256} \\ 8 & \frac{1}{256} & 1 & \frac{1}{256} \\ \end{array}$$ and for a given significance level $\alpha$, we find the largest value $x_L$ such that $\Pr[X \le x_L] \le \alpha/2$, and the smallest value $x_U$ such that $\Pr[X \ge x_U] \le \alpha/2$. If no such $x_L$ and/or $x_U$ exist, then the rejection region contains no values from the respective tail(s).
So for example, if $\alpha = 0.10$, then $\alpha/2 = 0.05$, and $\Pr[X \le 1] = \frac{9}{256} = 0.0351563$ is the largest left-tail probability that does not exceed $0.05$. Similarly, $\Pr[X \ge 7]$ is the largest right-tail probability. So the rejection region for this test is $X \in \{0, 1, 7, 8\}$; i.e., you would reject $H_0 : p = 0.5$ and claim the coin is unfair at the $10\%$ significance level if you observe no more than $1$ head or no less than $7$ heads out of $8$ tosses.
But what happens at $\alpha = 0.05$? Then your rejection region shrinks to $X \in \{0, 8\}$. This makes intuitive sense; the smaller the $\alpha$, the higher the burden of evidence needed to conclude the coin is unfair.
And what if $\alpha = 0.005$? Then your rejection region is the empty set, because the burden of evidence has become so high that $8$ tosses of the coin is insufficient to meet that standard, since $\Pr[X = 1] = \Pr[X = 8] = \frac{1}{256} > \alpha/2$.
What about the $p$-value of the test? For instance, what is the $p$-value of the observation $X = 7$ for this hypothesis? Well, it is simply $$p = \Pr[X \le 1] + \Pr[X \ge 7] = \frac{9}{128} = 0.0703125.$$
Note that in this entire discussion, we have not used any kind of approximation: all calculations are based on the exact binomial probabilities, and there is no "small sample" use of a $t$-statistic. Use of such approximation is not only inadvisable for such small $n$, but as you can see, it is simply unnecessary. The calculation is entirely tractable by hand and we do not even need to resort to statistical tables.