What does the conclusion with confidence interval mean, non-statistically speaking?

Question

There is a manufacturer who told me that they tested devices in the batch based on 95% confidence level, 5% confidence interval/margin of error and 50% distribution. So from 1500 devices, they tested 110. What does it mean for me? That I can be sure that there is 95% chance that all devices will behave as the tested samples?

Answer 1

Under the frequentist interpretation, the confidence interval is some interval $(a,b)$ where $a,b$ are random numbers, which are determined by the sample. For a $95\%$ confidence interval for estimating some quantity $\mu$, $95\%$ of the time you will have $a<\mu<b$. Note that in a fixed sample, either $a<\mu<b$ is true or it isn't. We never really know which. But under the frequentist interpretation we can say that if we collected many confidence intervals, $95\%$ of them would contain $\mu$ and $5\%$ would not contain $\mu$.

Note that under this interpretation it is not valid to say that $\mu$ is in our single interval $(a,b)$ with probability $0.95$, which is what you would naively expect. Under the Bayesian interpretation, something like this is indeed true, but there are subtleties involved in the Bayesian interpretation which are more difficult to explain.