Denormalization of numpy matrix is not correct

1.6k Views Asked by At

I have a $4D$-array of shape $(1948, 60, 2, 3)$ which I normalized to a range of $[0,1]$

a sample of how it looks is below:

original_mat = array([[[  3.93048840e-05,   7.70215296e-04,  
1.13865805e-03], [ 1.11679799e-04,  -7.04810066e-04,   1.83552688e-04]])

After normalization (x - x_min)/ (x_max - x_min)

predicted =   array([[ 0.19302673, -0.03372632, -0.23808828],
         [ 0.30002626, -0.71888705,  0.71468331]])

I fed this input to a neural network to predict a similar output, after convergence my resultant matrix looked the same and to de-normalize it, I did,

denormed_matrix = predicted*(xmax - xmin) + xmin
`denormed_matrix` = [[-0.62747524, -0.72737077,  0.70058271],
         [-0.39488326, -0.18533665, -1.48910199]],

I expected it to have same order of magnitude values ( e-03 to e-05), but the matrix didn't scale down in magnitude, it had similar values like the normalized one.

  1. Am I missing any point here?
  2. Are my calculation correct?

EDIT

CODE for Normalization

### Get min, max value aming all elements for each column
    

x = np.asarray(poseList)
    x_min = np.min(x, axis=tuple(range(x.ndim-1)), keepdims=1)
    x_max = np.max(x, axis=tuple(range(x.ndim-1)), keepdims=1)
    #
    ### Normalize with those min, max values leveraging broadcasting
    normalized = (x - x_min)/ (x_max - x_min)
    normalized = 2.0*normalized - 1.0    # noralizing in the range [-1,1]
    #
    print "final_save"
    
    In [75]: norm.shape
    Out[75]: (309, 60, 2, 3)
    
    In [16]: x_max
    Out[16]: array([[[[ 0.10778677,  0.16254221,  0.1198302 ]]]])
    
    In [17]: x_min
    Out[17]: array([[[[-0.56810854, -0.21604319, -0.37091526]]]])

Code for Denormalization
Following this formula $$X=\frac{(X_{max}-X_{min})(X'-a)}{b-a}+X_{min}$$

normalized = np.load('/home/normalized.npy')
normalized = normalized+ 1  #[a,b] = [-1,1]
diff = x_max - x_min
numerator = diff * normalized
denormalized = (numerator/2.0 ) + x_min
1

There are 1 best solutions below

4
On BEST ANSWER

The denormalization formula is wrong.

Normalization:

$X' = a + \frac{\left(X-X_{\min}\right)\left(b-a\right)}{X_{\max} - X_{\min}}$

Denormalization (inverse formula):

$X = \frac{(X_{\max}-X_{\min})(X'-a)}{b-a} + X_{\min}$


Example with original_mat(0,0):

$X' = 2\frac{(3.93048840 + 7.04810066)}{(7.70215296 + 7.04810066)}-1$

$X' = 2\frac{10.97858906}{14.75025362}-1$

$X' = 0.48859665$

denormalization:

$X = \frac{14.75025362(0.48859665+1)}{2} - 7.04810066$

$X = \frac{21.95717813}{2} - 7.04810066$

$X =3.93048840$

I hope I have been of help, best regards,

Marco.