I am getting confused when trying to determine the statistical significance of my results. For instance, I was attempting to make a conclusion of reliability of a model after 100000 trials depending on "the number of errors" coming after each trial. I found the probability of getting 1,2,3,4,5,6 errors as follows (all are independent):
P(1)= 0.133
P(2)= 0.400
P(3)= 0.200
P(4)= 0.200
P(6)= 0.067
Now I want to deternime the statistical significance of the above results at 1% level. What does it literally mean?
Does it mean that,
In case of 1 error:- One error has come at least 1% of total trials i.e. at least 1000 times?
In case of 2 errors:- Two errors have come at least 1% of total trials i.e. at least 1000 times?
And so on...
Can anybody suggest/clarify something in this regard.
In your question, you haven't really given us much information about the model(s) underlying your analysis. We can't, for example, tell whether $2$ or $3$ errors is a surprising outcome. to decide things like that, we'd need to know about the model.
To your general question. Statistical Significance arises when you have a model (such as "I assume this coin I'm holding is fair and therefore just as likely to come up H as T when I toss it"). You then go ahead and toss it ( a lot). and now you can ask "is it probable that, given my assumption (i.e. that the coin was fair) I'd get this data?" Of course the data may be possible, but the question is whether or not it is probable. Informally, if you toss the coin $100$ times and get exactly $49$ Heads, that is hardly a shocking result and we;d declare the result Statistically Insignificant. On the other hand, if you got $96$ Heads out of $100$ tosses then you'd surely have a Statistically Significant result (and excellent cause to reject the notion that the coin was fair).
More formally, Before you toss the coin, you should declare a confidence level. You mentioned $1\%$ so let's use that. What does that mean? Well, I'll be counting Heads so I establish the band $[50-N,50+N]$ in which $99\%$ of the experiments should fall. In this case, taking $N=13$ is sufficient. To summarize: The probability that the number of Heads observed in $100$ tosses of a fair coin lies in the band $[37,63]$ is greater than $99\%$. Then, if you do your experiment and get $55$ Heads, you can conclude nothing. On the other hand, if you only observe $28$ Heads then you can reject the hypothesis (that the coin was fair), again within the $1\%$ confidence level.
The hard part, almost always, is computing the theoretical probabilities. In the example I just gave, I used a normal distribution to approximate the fair coin. In more complex models, it can be very unclear how to compute what you need. In practice, one often just simulates the model on a computer and estimates the probability that way.