$X$ and $Y$ are two continuous variables, where $Y=X^2$. How can we find $\text{covariance}(X,Y)$?
So, I start:
$\text{cov}(X,Y) = E_{XY}[XY] - E[X]E[Y]$
$= E_{XY}[XX^2] - E[X]E[X^2]$
Then I am thinking of dividing everything by $E[X]$ to get rid of power but then it becomes $0$. Also, I don't know how to solve $E_{XY}[XX^2]$? Is it just $E_{XY}[X^3]$? Can you help me out here?
It might be instructive to look at this question from the point of view of probability density functions (pdf's). Let the pdf of $X$ be denoted by $p_X(x)$: it is a nonnegative function, defined on a support $\sigma_X$ included in $\mathbb{R}$, such that $\int_a^b dx\ p_X(x)$ is the probability that the random variable $X$ takes a value between $a$ and $b$. As such, it needs to be normalized, i.e. $\int_{\sigma_X}dx\ p_X(x)=1$ (as the random variable $X$ will take with probability $1$ one value inside its support). Two (or more) random variables are described by a joint probability density function (jpdf) $p_{X,Y}(x,y)$, such that the integral $\int_{a}^b dx\int_c^d dy\ p_{X,Y}(x,y) $ denotes the probability that $X$ takes a value between $a$ and $b$, and $Y$ a value between $c$ and $d$. If $p_{X,Y}(x,y)=p_X(x)p_Y(y)$ the two random variables are called independent, otherwise they 'talk to each other'. Clearly, if $Y=X^2$ as in our case, the two random variables $X$ and $Y$ are not independent. What is their jpdf? [In the following, I will assume that $\sigma_X=(-\infty,\infty)$]. One may write $$ p_{X,Y}(x,y)=p_X(x)\delta(y-x^2)\ , $$ where the Dirac delta ensures that values of $Y$ different from $x^2$ have probability zero. As a check, we see that the marginal pdf's $$ p_X(x)=\int_0^\infty dy p_X(x)\delta(y-x^2)=p_X(x) $$ and $$ p_Y(y)=\int_{-\infty}^\infty dx p_X(x)\delta(y-x^2)=\frac{p_X(\sqrt{y})+p_X(-\sqrt{y})}{2\sqrt{y}}\ . $$ Note that the jpdf is correctly normalized. Now, we can compute the sought covariance, by computing first the expectation values $$ \mathbb{E}_{X,Y}[XY]:=\int_{-\infty}^\infty dx\int_0^\infty dy\ xy\ p_{X,Y}(x,y)=\int_{-\infty}^\infty dx\int_0^\infty dy\ xy\ p_X(x)\delta(y-x^2)\ , $$ and performing the $y$-integral first, $$ \mathbb{E}_{X,Y}[XY]=\int_{-\infty}^\infty dx\ x\ p_X(x)x^2=\int_{-\infty}^\infty dx\ x^3\ p_X(x)=\mathbb{E}_X[X^3]\ . $$ The other expectation values are easy: $$ \mathbb{E}_{X,Y}[X]=\mathbb{E}_{X}[X]=\int_{-\infty}^\infty dx p_X(x)x $$ $$ \mathbb{E}_{X,Y}[Y]=\mathbb{E}_{Y}[Y]=\int_0^\infty dy p_Y(y)y=\int_0^\infty dy\ y\frac{p_X(\sqrt{y})+p_X(-\sqrt{y})}{2\sqrt{y}}\ , $$ which (changing variable $\sqrt{y}=t$) implies $\mathbb{E}_{Y}[Y]=\mathbb{E}_{X}[X^2]=\int_{-\infty}^\infty dx\ x^2 p_X(x)$. Without specifying the actual $p_X(x)$, we cannot proceed much further than that. Note that the integral representations of the various expectation values make it clear that we cannot 'simplify' anything: operations like $\frac{\mathbb{E}[X\cdot X^2]}{\mathbb{E}[X]}\stackrel{?}{=}\mathbb{E}[X^2]$ are obviously not legitimate.