Would a T test be appropriate for determining the significance of an algorithm's results?

Question

Say you have an algorithm which should yield a higher score when given two similar images and a lower value when given two images that do not resemble each other. Two sets of tests are run, one where pairs of similar images are given and the other where pairs of different images are given. Each test set contains 10 tests on complex images. The algorithm yields a higher score, on average, for the images which resemble each other. Although this seems to confirm a hypothesis that the algorithm works, statistical significance needs to be calculated. Would a T test be appropriate for this? If so, should the normal distribution of results from both test sets first be proven?

Answer 1

Comment continued: Here are examples--perhaps helpful. You have said very little about your data--maybe not.

Two sample Welch t test in R: P-value near $0;$ highly significant difference between sample mean. (Difficult to judge normality with such small samples. I generated these 'fake' samples from normal distributions, so I'm sure they're normal.)

set.seed(102)
x = rnorm(10, 100, 5)
y = rnorm(10, 120, 6)
stripchart(list(x,y), ylim=c(.5,2.5), pch="|")

t.test(x, y)  

        Welch Two Sample t-test

data:  x and y
t = -5.5749, df = 15.849, p-value = 4.334e-05
alternative hypothesis: 
 true difference in means is not equal to 0
95 percent confidence interval:
   -20.854715  -9.357465
sample estimates:
mean of x mean of y 
 103.8186  118.9247 

Wilcoxon Signed Rank test in R: P-value near $0;$ highly significant difference between sample medians.

set.seed(121)
x = rgamma(10, 3, .1)
y = rgamma(10, 3, .1) + 20

median(x); median(y)
[1] 21.53694
[1] 54.64572

stripchart(list(x,y), ylim=c(.5,2.5), pch="|")

wilcox.test(x, y)

        Wilcoxon rank sum test

data:  x and y
W = 5, p-value = 0.0002057
alternative hypothesis: 
  true location shift is not equal to 0