Background: We are doing mathematical transformations on large matrices of inline fraunhofer holography data (the nature of this isn't really important, but if you are curious, essentially it is a forward fast Fourier transformation into complex values with a little bit of filtering done to prepare the data, and then an inverse FFT to take the values back to real values). The transformation process is done to unpack single slices of data from the holographic data traveling in the Z direction in order to do analysis on each image slice of the overall hologram.
For those interested: The hologram data is created by recording the light from a plane-wave which is passed through the holographic film (cubic medium) and recording the output into a camera.
The entire formula is being calculated in IDL (a similar program to MatLab) and is as follows (again, this isn't the main idea of the question but you might find it of interest): U(ksi, eta) = FFT^{-1}{ FFT{h(x,y)}*EXP{-i*2*Pi*z/lambda * sqrt( 1 - lambda^2*( fx^2 + fy^2 ) ) } }
Where:
h(x,y) = h0 + beta * tau * I(x,y)
h(x,y) - amplitude transmittance of the developed photographic plate
beta - constant, slope of the transmittance vs exposure curve
tau - exposure time
h0 - amplitude transmittance of the unexposed plate
I(x,y) - intensity of the image
Problem: The specifics of the transformation of the data aside. We decided to transport the process to C++ code in order to do the calculations more quickly, by doing it on better hardware and in parallel.
So we have a test series of 100 image slice outputs from both programs. These images are basically double floating point values in a matrix.
Given: A is one of the 100 output matrices of data values from the IDL program, B is the matching one of the 100 output matrices of data values from the C++ program, C is the difference matrix C = A - B
A histogram is created from each C difference matrix and a histogram is generated from all the values contained in the the C matrix. The expected values of each difference matrix should be random values that follow closely to a normal distribution, this would show that the output from the IDL program and the output from the C++ program are identical except for a floating point error.
However, after evaluating these 100 iterations of C difference matrices, the sigma values (Standard Deviations) of each of the 100 histograms of C becomes larger at a linear rate; around apx +/- 10 bins of the median at the 0th image, to apx +/- 500 mins of the median value at the 100th image.
From this result, is this sufficient evidence that the process is significantly broken in the C++ output, enough to justify that the C++ version of the mathematical transformations is not reliable?
a.k.a. Is an increase from a sigma of 10 to a sigma of 500, and an overall flattening of the normal distibution over the 100 frames, enough evidence to support the C++ being massively overhauled?