Test Statistic for Comparing Two Mutually Exclusive Sub-Sample Means

40 Views Asked by At

I am trying to compare the month-over-month percentage changes in currency returns under various financial market conditions. I have 30 years of monthly currency data (N=360) and am using the VIX index to capture volatility in markets.

I calculated the average month-over-month change in the value of the currency when volatility is above some threshold and when it is below. I would like to compare these means to see if there is a statistically significant difference. I would also like to compare the average return in high volatility conditions to 0%. I'm not sure the appropriate test statistic or degrees of freedom to use. Would I use a two-sample t-test or just a one-sample, or something else entirely?

Also, when I set a high threshold for volatility, there are less than 10 observations that can be used to calculate the "high-volatility" mean. Will this significantly increase the critical value used in tests compared to lower thresholds where there are more like 15-25 applicable months?

1

There are 1 best solutions below

0
On

For Hi vs Lo volatility I would suggest a two-sample test. Check the two samples separately to see if the observations are nearly normal [formally with something like a Shapiro-Wilk test of normality; informally by looking a normal probability plots (quantile-quantile plots)]. If nearly normal, use the Welch (separate-variances) version of the two-sample t test. If not normal, you should use some kind of nonparametric two-sample test. (Possibilities include Wilcoxon rank sum test, a permutation test, or an approximate t test on the ranks of the observations. Perhaps show some of your data if you need help deciding among these.)

To compare average return to 0 at high volatility, you would need a one-sample test. One-sample t if observations are normal, perhaps Wilcoxon signed rank test if not.


The degrees of freedom (DF) for a two-sample Welch t test are determined by a moderately messy formula that involves sample sizes and sample standard deviations. Most statistical software will find DF as part of the Welch procedure.

If I understand correctly, you may have 'fewer than 10' observations at high volatility. I'm wondering how this number got so small considering you start out with 360 months. If you have as few as five, then rank-based nonparametric tests will not be able to give significant results. Also, if your sample size for Hi in a two-sample test are as small as 10 and variances in the Hi and Lo groups are extremely different, the Welch procedure may not be able to give a significant result. In those circumstances, it seems best to change the threshhold so you'll have nearer to 20 Hi observations.

It is always difficult to give definitive advice what tests to use without seeing the data. If you run into problems deciding which test to use or problems making the tests work, and if you are comfortable showing (at least some of) your data, one of us might be able to give further guidance.