Could anyone please explain in a simple manner why do I get different results or point to the mistake I make? Do these calculations even make sense given the sample sizes and binomial outcomes?
I have two independent equal binomial samples and I want to test whether the observed difference between them is significant.
$n_1$ = $n_2$ = 112
"Success rate" in the first sample: $\hat p_1$ = 55.4
"Success rate" in the second sample:t: $\hat p_2$ = 46.4
Hence, I test the following hypothesis: $H_0:p_1=p_2$ versus $H_A:p_1\neq p_2$ by using the following formula:
$$z=\frac{\hat p_1-\hat p_2}{\sqrt{\hat p(1-\hat p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}}$$
where $\hat p=\frac{n_1\hat p_1+n_2\hat p_2}{n_1+n_2}$.
I get that the $z$ = 2.69 which is larger than $z_{\alpha/2}=1.96$. Hence, I can reject the null hypothesis with 95% confidence.
However, if I compute CI for a difference in proportions using this formula:
$$\hat p_1 - \hat p_2±z\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}$$
I get that the difference falls into CI of $(-0.04,0.22)$ which suggests that 9 p.p. difference between $\hat p_1$ and $\hat p_2$ is actually insignificant given that $0$ is in the CI.
Thanks for your insights.
Looks like you made a mistake in calculating $z$ in the first part. With $\hat p=0.509$, I get
$$ z=\frac{0.554-0.464}{\sqrt{0.509(1-0.509)\left(\frac1{112}+\frac1{112}\right)}}\approx1.35\;. $$