I have two data sets $M$ and $N$ that are equally large. They have different averages and medians and I want to prove that, with 5% significance, the values don't differ solely out of chance.
I'm trying to understand what the two-tailed paired t-test really tells me. I've performed such a test using Google Sheets built-in function TTEST() and it returns the value 0.03. Does this mean that it's a 3% chance that the values differed as they did solely out of chance and that I've successfully proved (at a 5% significance level) that they in fact differ?
Yes. This is the so-called $p$-value. And if you get $0.03<0.05$ then the hypothesis that the means are equal should be rejected at $5\%$ significance level since there is only $3\%$ of chances that data sets with the same means show such difference of averages.