Correlation coefficient plus scaling and shifting

Question

I am training a convolutional neural network with the Pearson correlation coefficient $\rho(a,b)$ where the error function $\epsilon$ is defined as $1-\rho$, for a given predicted output $a$ and a target output $b$ ($a$ and $b$ are images).

As expected, the result is that the predicted image $a$ correlates well with $b$, but there is a scaling and shifting difference between the two images (for example, $a=1+2b$).

My question is how to fix the scaling and shifting issue within the error function. Would including the difference between the means (fix shift) and the standard deviations (fix scaling) work? So the final error function could be something like:

$\epsilon(a,b) = (1-\rho(a,b))+(\bar{a}-\bar{b})^2+(\sigma_a-\sigma_b)^2$

Answer 1

No, that approach won't work because it's dimensionally inconsistent. I'll try to motivate a more sensible choice, but we can't "prove" some formula is "correct", only explain why one would be appropriate.

Since $\rho(a,\,b)=\frac{\sum_i (a_i-\overline{a})(b_i-\overline{b})}{\sqrt{\sum_i (a_i-\overline{a})^2\sum_i (b_i-\overline{b})^2}}$, we can penalise the variances $\frac{1}{n}\sum_i (a_i-\overline{a})^2,\,\frac{1}{n}\sum_i (b_i-\overline{b})^2$ of $n$ data points differing by replacing their geometric mean in the denominator with their arithmetic mean. We can also penalise differing means. Since $$\left(a_i-\frac{\overline{a}+\overline{b}}{2}\right)\left(b_i-\frac{\overline{a}+\overline{b}}{2}\right)-(a_i-\overline{a})(b_i-\overline{b})=\frac{(\overline{a}-\overline{b})^2}{4}-\frac{\overline{a}-\overline{b}}{2}(a_i-b_i),$$ we have $$\sum_i\left[\left(a_i-\frac{\overline{a}+\overline{b}}{2}\right)\left(b_i-\frac{\overline{a}+\overline{b}}{2}\right)-(a_i-\overline{a})(b_i-\overline{b})\right]=-n\frac{(\overline{a}-\overline{b})^2}{4}\le 0.$$So we can reduce our expression for $\rho$ further by sharing an averaged mean. In other words, the final formula for $\epsilon$ could be $$\epsilon=1-2\frac{\sum_i (a_i-\frac{\overline{a}+\overline{b}}{2})(b_i-\frac{\overline{a}+\overline{b}}{2})}{\sum_i (a_i-\overline{a})^2+\sum_i (b_i-\overline{b})^2}.$$Or even better, we can use$$\epsilon=1-2\frac{\sum_i (a_i-\frac{\overline{a}+\overline{b}}{2})(b_i-\frac{\overline{a}+\overline{b}}{2})}{\sum_i (a_i-\frac{\overline{a}+\overline{b}}{2})^2+\sum_i (b_i-\frac{\overline{a}+\overline{b}}{2})^2},$$which has the advantage we only need to calculate one translated copy of each of the two original datasets.

Here's how you can code such a function in Python:

from numpy import array, mean
def epsilon(ab_pairs):
    a, b = [array(x) for x in zip(*ab_pairs)]#a, b are 1D arrays
    c = (mean(a)+mean(b))/2
    a, b = a-c, b-c
    return 1-2*a.dot(b)/(a.dot(a)+b.dot(b))