I'm getting the hang of using the properties of Covariance to make calculating it much easier but I'm stuck on this one.
Fair coin tossed $10$ times. Let $X$ denote number of heads observed and $Y=X^2$. Find Covariance.
Well the $E(X)=5$ heads and $E(Y)=25$. How could I calculate $E(XY)$. At first I thought it would just be $E(X)^3$ which equals $125$ so the covariance would be $0$ but this wasn't the right answer.
On
Now $\mathrm{Cov}(X,Y)=\mathrm{E}(XY)-\mathrm{E}(X)\cdot\mathrm{E}(Y) = \mathrm{E}(X^3) - \mathrm{E}(X)\mathrm{E}(X^2)$
This should not be $0$, as the value of $Y$ is clearly not going to be independent of the value of $X$.
Let $X_i$ be the event of the $i^{th}$ toss turning up a head, for $i=1..10$. Each $X_i$ has a possible value of either $0$ or $1$.
So this gives: $X=X_1+X_2+X_3+X_4+X_5+X_6+X+7+X_8+X_9+X_{10}$ which we can write as $X=\sum\limits_{\forall i}X_i$
Obviously for one coin: $\mathrm{E}(X_i) = \frac 12 (1+0) = \frac 12$, but also $\mathrm{E}(X_i^2) = \frac 12 (1^2+0^2) = \frac 12$.
However, when the coins are different, there are 4 possible outcomes, so $\mathrm{E}(X_i X_j \mid i\neq j) = \frac14(1\cdot 1 + 1\cdot 0 + 0\cdot 1+0\cdot 0) = \frac 14$
This matters because $X^2 = (\sum\limits_{\forall i} X_i)(\sum\limits_{\forall j} X_j) = \sum\limits_{\substack{\forall i, j\\ i=j}} X_iX_j + \sum\limits_{\substack{\forall i,j\\i\neq j}} X_iX_j = \sum\limits_{\forall i} X_i^2 + \sum\limits_{\substack{\forall i,j\\i\neq j}} X_iX_j $
So we can use the expected values of individual coins to determine the expected values of $X$, $X^2$, and $X^3$ which, not coincidentally, gives us the expectation for $Y$ and $XY$.
$\mathrm{E}(X) $ $= \mathrm{E}(\sum\limits_{\forall i} X_i) \\ = \sum\limits_{\forall i} \mathrm{E}(X_i) \\ = 10\cdot\frac 12 \\ = 5$
$\mathrm{E}(Y) = \mathrm{E}(X^2)$ $= \mathrm{E}((\sum\limits_{\forall i} X_i)(\sum\limits_{\forall j} X_j)) \\ = \mathrm{E}(\sum\limits_{\forall i} X_i^2) + \mathrm{E}(\sum\limits_{\substack{\forall i,j\\ i\neq j}} X_iX_j) \\ = 10\cdot\frac 12 + 10\cdot 9 \cdot \frac 14 \\ = \frac {55}2$
$\mathrm{E}(XY)=\mathrm{E}(X^3)$ $= \mathrm{E}((\sum\limits_{\forall i} X_i)(\sum\limits_{\forall j} X_j)(\sum\limits_{\forall k} X_k)) \\ = \mathrm{E}(\sum\limits_{\forall i} X_i^3) + 3\mathrm{E}(\sum\limits_{\substack{\forall i,k \\ k\neq i}} X_i^2X_k) + \mathrm{E}(\sum\limits_{\substack{\forall i,j,k\\ i\neq j\neq k\neq i}} X_iX_jX_k) \\ = 10\cdot\frac 12 + 3\cdot 10\cdot 9\cdot\frac 14 + 10\cdot 9 \cdot 8 \cdot \frac 18 \\ = \frac {325}2 $
Thus $\mathrm{E}(X^3)\neq \mathrm{E}(X)^3$ and likewise $\mathrm{E}(X^2)\neq \mathrm{E}(X)^2$.
Put it all together to find $\mathrm{Cov}(X,Y) = \mathrm{E}(X^3) - \mathrm{E}(X)\mathrm{E}(X^2) = \frac{325}2 - 5\cdot\frac{55}2 = 25$
You want $E(XY)-E(X)E(Y)$, which is $E(X^3)-E(X)E(X^2)$. You know how to deal with the last part, so you need $E(X^3)$. The value of this is not obvious. In the main answer we look at as general approach, and in a remark at the end mention a crude way to do the calculation.
You can find $E(X^3)$, indeed $E(X^n)$, by using the moment generating function of the binomial. This is the function $M_X(t)$ defined by $$M_X(t)=E(e^{tX}).$$ That is equal to $$\sum_0^{10} e^{tk}\binom{10}{k}\left(\frac{1}{2}\right)^{10}.$$ Note that $e^{tk}=(e^t)^k$, so our sum is $$ \left(\frac{1}{2}\right)^{10}\sum_{k=0}^{10} \binom{10}{k}(e^t)^k.$$ By the Binomial Theorem, we get $$M_X(t)=\left(\frac{1}{2}\right)^{10}(e^t+1)^{10}.$$ Now you can get the expectation of $X^3$ by finding the third derivative of the mgf at $t=0$.
Remark: For your particular example, you can find $E(X^3)$ more directly by calculating the sum $$\sum_{k=0}^{10} k^3\binom{10}{k}\left(\frac{1}{2}\right)^{10}.$$ Not much fun, but doable.