I am trying to get a refresher on basic statistical theory and I found myself in an awkward position.
The definition of covariance from wikipedia is:
$cov(X,Y) = E[X,Y] - E[X]E[Y]$
Ok fair enough, I know how to calculate the right hand side, but I am not sure what to do with the left.
So then I searched for "expected value of product of random variables"
And that returned: $E[XY] = cov(X,Y) + E[X]E[Y]$
Which isn't particularly helpful, I am also able to reorder the terms of an equation.
My goal is merely to understand the definition of covariance in a way I can compute it. But I have not been able to find a resource that defines the expected value of a product of random variables without relying on the definition of covariance.
The definition for covariance of two random variables, is that it is the expected product of their displacements from their means.
$$\mathsf{Cov}(X,Y)=\mathsf E((X-\mathsf E(X))\,(Y-\mathsf E(Y)))$$
Which gives us that: $$\begin{align}\mathsf{Cov}(X,Y)&=\mathsf E(XY-X\,\mathsf E(Y)-Y\,\mathsf E(X)+\mathsf E(X)\,\mathsf E(Y))\\&=\mathsf E(XY)-\mathsf E(X\,\mathsf E(Y))-\mathsf E(Y\,\mathsf E(X))+\mathsf E(\mathsf E(X)\,\mathsf E(Y))\\&=\mathsf E(XY)-\mathsf E(X)\,\mathsf E(Y)-\mathsf E(Y)\,\mathsf E(X)+\mathsf E(X)\,\mathsf E(Y)\\&=\mathsf E(XY)-\mathsf E(X)\,\mathsf E(Y) \end{align}$$
The covariance of a random variable and itself is called the variance.$$\begin{split}\mathsf{Var}(X)&=\mathsf{Cov}(X,X)\\&=\mathsf E((X-\mathsf E(X))^2)\\&=\mathsf E(X^2)-\mathsf E(X)^2\end{split}$$
The usefulness of covariance is that comparing it to the product of the variances (more specifically to the square root of that product) gives a measure for how linearly dependent the two random variables may be. This is the correlation coefficient.
$$\mathsf{Corr}(X,Y)=\dfrac{\mathsf{Cov}(X,Y)}{\surd(\mathsf{Var}(X)\,\mathsf{Var}(Y))}$$