Why is Covariance defined as:
- $\operatorname{cov}(X,Y) = E[(X-E[X])(Y-E[Y])]$
My book simply states this identity but doesn't explain how it is derived. I know that covariance can also be written as:
- $\operatorname{cov}(X,Y) = E[XY] -E[X]E[Y]$
I know how to derive this identity from the first definition.
I don't know if there is any technical reason behind this identity. There is a simple intuition behind the identity. We want to know if the variables X and Y are linearly related or not.i.e, can we say something like Y= m X where m is a constant. If m is positive, then Y increases as X increases. If m is negative, the Y decreases as X increases. First we take rescale both the variables around their mean by looking at P=X-E[x] and Q=Y-E[Y]. If P and Q are both positive or negative, then it adds to the covariance. If one of them is positive and the other is negative ,it subtracts from the covariance. Finally, the expectation gives the average behaviour.