I'm doing an assignment, that envolves timing multiple encryption/decryption algorithms, and I'm having trouble plotting them.
My data is something in the lines of:
| Bytes | Mean (ms) | Confidence Interval Offset |
|---|---|---|
| 8 | 17.778873443603516 | 0.7680876633930153 |
| 64 | 17.147064208984375 | 0.4299001276011886 |
| 512 | 22.0489501953125 | 2.150226687050264 |
| 4096 | 27.785301208496094 | 1.6384117673799095 |
Like it says in the table, I calculated the Confidence Interval (95%) for the data, but I'm not sure if this is completely correct. Do I need to make sure that the data represents a normal distribution? If I need to, how do I go about doing that?
The data in the table is after I ran 100 randomized trials for each entry.
Are confidence intervals even appropriate here?
The most common formula used for a 95% confidence- that is the sample mean plus or minus 1.96 times the standard error- is valid only when the sample mean has a normal distribution and the standard error is known. If the standard error is estimated from the data, then the 1.96 should be replaced by a quantile of the t-distribution. If the sample size is large, that quantile will be approximately 1.96. If the sample is not normally distributed, there is no general fix to it. One thing I can think of is using the bootstrap method.
You can investigate whether the distribution is normal by looking at the 100 randomized trials for a fixed number of Bytes. Look at the histogram and the normal probability plot.
Three suggestions:
You may want to switch to the sample median if the distribution is skewed.
You many want to increase the number of trials to something larger if you can. If you can only do that for the smaller number of Bytes, that's better than nothing.
Plot the Bytes on the x-axis and the mean (or median) on the y-axis. Look to see if you can guess what the relationship might be. With the data you have now, the number of trials is too small and the number of different Bytes (4) is also too small to guess at what the relationship might be. Can you try others?