Is this claim justified at the $95\%$ confidence interval?

123 Views Asked by At

$400$ samples were tested out of batch $A$ and $88\%$ passed the test.

$1000$ samples were tested out of batch $B$ and $810$ passed the test.

The company claimed that at the $95\%$ confidence level there was no difference between these two batches.

Is the claim justified at the $95\%$ confidence level?

My textbook says no, but I disagree.

The textbook calculates the ME from: $ME=\frac{1}{\sqrt{n}}$

From batch $A$ the $95\%$ confidence interval is $88.0 \pm 5.0\%$.

My textbook says that the sample proportion for batch $B$ (81%) doesn't not fall inside this interval so it is not justified.

But my argument is that batch $B$ has it's own $95\%$ confidence interval of $81 \pm 3.2\%$.

The confidence intervals have an overlap at the $95\%$ level so I would argue that the claim is justified.

Which approach is correct ?

1

There are 1 best solutions below

2
On

The reasoning given in the textbook is wrong, and the reason why it is wrong makes me question whether this text is suitable for use.

When we construct a univariate confidence interval, we are making a statement only about the data that generated that interval. You cannot use it to make an inference about a second sample that was not used to construct that interval.

The correct way to answer the question is to conduct a hypothesis test for two independent proportions; e.g., $$H_0 : p_a = p_b \quad \text{vs.} \quad H_1 : p_a \ne p_b,$$ where $p_a$ and $p_b$ are the true proportions of passing for batch $A$ and $B$, respectively. An asymptotically normal test statistic would be $$Z = \frac{\hat p_a - \hat p_b}{\sqrt{\hat p (1 - \hat p) (n_a^{-1} + n_b^{-1})}},$$ where $\hat p_a$, $\hat p_b$ are the observed proportions in each batch, $$\hat p = \frac{n_a \hat p_a + n_b \hat p_b}{n_a + n_b}$$ is the pooled proportion of passing across both batches, and $n_a$, $n_b$ are the sample sizes in each batch.

Alternatively, you can conduct a chi-squared test using a $2 \times 2$ table. The resulting test statistic will be chi-squared with one degree of freedom.

If you want to use a confidence interval approach, then you must construct a confidence interval for the difference in proportions between the two batches, and if this confidence interval does not contain $0$ at the $95\%$ level, then you can assert that there is a statistically significant difference in the passing proportions between the two batches. But you absolutely cannot make this assertion by comparing the sample proportion in Batch B against the interval estimate calculated from Batch A. This is so completely and obviously wrong that any textbook claiming you can do this should, in my opinion, be thrown out and ridiculed.